1
$\begingroup$

I'm trying to supply knitro with a Hessian but struggle to understand the dimension of the Lagrangian multiplier $\lambda$. From my general education and knitro's documentation which gives the Lagrangian as $$ \mathcal{L}(x,\lambda,\sigma) := \sigma f(x) +\sum\limits_{i=0}^{m-1} \lambda_i c_i(x), $$ where $f$ is the objective and $c_i(x)$ are the constraints, I would think that $m$ is the number of constraints. When I implemented my Hessian (as function of $x$, $\sigma$ and $\lambda$, I noticed that the Hessian is evaluated in an unexpected way because the dimension of $\lambda$ seemed to equal that of $x$ although I have much more constraints than variables. That is why the Hessian returns nan when calling knitro although the Hessian evaluates normally when I supply $x$, $\lambda$ and $\sigma$ with the expected dimensions.

This is also the case in the examples provided by knitro. For example, in the example HS71 in HS71.R, they show how to solve the problem $$ \min\limits_{x1,x2,x3,x4} x_1 x_4 (x_1 + x_2 + x_3) + x_3 \quad \text{ s.t. } \begin{cases} x_1 x_2 x_3 x_4 \ge 25,\\ x_1^2 + x_2^2 + x_3^2 + x_4^2 = 40,\\ 1 \le x_1, x_2, x_3, x_4 \le 5. \end{cases} $$ In the supplied code for the Hessian in this case, eval_h(x, lambda, sig), Artelys uses lambda[1] and lambda[2], i.e. $\lambda$ is two-dimensional, as I would expect. However, when knitro is called, i.e. res <- knitro(...), then

res$lambda = c(-1.087872e+00,  2.742766e-08, -9.083440e-09, -1.797043e-07)

which is suddenly four-dimensional. I could imagine that the nonlinear constraints (of which there are 2) count double internally because they come with an upper and lower bound (although the upper bound for the first one if Inf).

In HS40.R, there are 3 constraints (suggesting that lambda would be six-dimensional). But lambda is only four-dimensional as per the output of knitro. I could imagine that the third inequality does not count as it is quadratic and knitro handles the constraints internally.

But in all these supplied examples, the number of constraints is lower than the number of variables and the dimension of lambda is equal to the dimension of x. In my application, there are more constraints than variables, but the dimension of lambda that knitro calls my Hessian with is also exactly the dimension of x. This leads to problems because I am referencing lambdas with "higher" indices.

To summarize, my questions are

  1. What exactly is m in the Lagrangian of knitro?
  2. Does a constraint like $l \le c_1 \le u$ count twice for the dimension of lambda?
  3. Does a linear or quadratic constraint count into m?
  4. Is the lambda in the documentation of the Hessian and the lambda in the output of knitro perhaps something else? (The documentation says that the output lambda is a "Vector containing the variable dual values." I thought of these as dual values of constraints, but knitro says specifically variable dual values).
  5. Is there a way to enforce the dimension of lambda for knitro's calls to the Hessian (I do supply ncon in my code and knitro correctly recognizes the number of constraints (not double-counting upper and lower bounds), but still supplies lambdas of a lower dimension than what I expect.).

Edit: If that is relevant, I am using knitro version 13.0.1.

$\endgroup$
3
  • $\begingroup$ The interface and Knitro function you are using are relevant. Note that only lambda values (Lagrange multipliers) corresponding to nonlinear constraints are used in Hessian evaluation (one for each inequality or equality constraint). $\endgroup$ Sep 8, 2023 at 10:38
  • $\begingroup$ I am using the R interface and the knitro function as in the provided example HS40.R or HS71.R. $\endgroup$
    – Frank
    Sep 8, 2023 at 14:24
  • $\begingroup$ You're more likely to get a definitive answer at groups.google.com/g/knitro . $\endgroup$ Sep 8, 2023 at 14:30

1 Answer 1

1
$\begingroup$

Indeed, there is a bug in the Knitro R interface. The size of the lambda array in the callback input is wrong, and the lambda array returned at the end only contains the dual variable values.

Here is a fixed version of the interface

Now, in the output, there is one lambda array for the constraint dual values and another lambdaVar array for the variable dual values.

$ Rscript HS40.R

=======================================
          Commercial License
         Artelys Knitro 13.2.0
=======================================

Knitro presolve eliminated 0 variables and 0 constraints.

concurrent_evals:        0

Problem Characteristics                                 (   Presolved)
-----------------------
Objective goal:  Maximize
Objective type:  general
Number of variables:                                  4 (           4)
    bounded below only:                               0 (           0)
    bounded above only:                               0 (           0)
    bounded below and above:                          0 (           0)
    fixed:                                            0 (           0)
    free:                                             4 (           4)
Number of constraints:                                3 (           3)
    linear equalities:                                0 (           0)
    quadratic equalities:                             0 (           0)
    gen. nonlinear equalities:                        3 (           3)
    linear one-sided inequalities:                    0 (           0)
    quadratic one-sided inequalities:                 0 (           0)
    gen. nonlinear one-sided inequalities:            0 (           0)
    linear two-sided inequalities:                    0 (           0)
    quadratic two-sided inequalities:                 0 (           0)
    gen. nonlinear two-sided inequalities:            0 (           0)
Number of nonzeros in Jacobian:                       7 (           7)
Number of nonzeros in Hessian:                        9 (           9)

Knitro using the Interior-Point/Barrier Direct algorithm.

  Iter      Objective      FeasError   OptError    ||Step||    CGits
--------  --------------  ----------  ----------  ----------  -------
       0    4.096000e-01   2.880e-01
       3    2.500000e-01   1.863e-10   2.661e-09   1.834e-05        0

EXIT: Locally optimal solution found.

Final Statistics
----------------
Final objective value               =   2.50000000082290e-01
Final feasibility error (abs / rel) =   1.86e-10 / 1.86e-10
Final optimality error  (abs / rel) =   2.66e-09 / 2.66e-09
# of iterations                     =          3
# of CG iterations                  =          0
# of function evaluations           =          4
# of gradient evaluations           =          4
# of Hessian evaluations            =          3
Total program time (secs)           =       0.16799 (     0.062 CPU time)
Time spent in evaluations (secs)    =       0.06291

===============================================================================

Final results:
$status
[1] 0

$obj
[1] 0.25

$x
[1] 0.7937005 0.7071068 0.5297315 0.8408964

$lambda
[1] -0.5000000  0.4719372 -0.3535534

$lambdaVar
[1] 0 0 0 0

$objGrad
[1] 0.3149803 0.3535534 0.4719372 0.2973018

$objGradIndexVars
[1] 0 1 2 3

$c
[1] 1.000000e+00 9.385481e-11 1.863212e-10

$r
NULL

$jac
[1]  1.8898816  1.4142136  1.3348399 -1.0000000  0.6299605 -1.0000000  1.6817928

$jacIndexCons
[1] 0 0 1 1 1 2 2

$jacIndexVars
[1] 0 1 0 2 3 1 3

$rsdJac
NULL

$rsdJacIndexRsds
NULL

$rsdJacIndexVars
NULL

$numFCevals
[1] 4

$numGAevals
[1] 4

$numHevals
[1] 3

$numHVevals
[1] 0

$numIters
[1] 3

$numCGiters
[1] 0
$\endgroup$
4
  • $\begingroup$ Outstanding, thank you! Never suspected a bug in the software. $\endgroup$
    – Frank
    Sep 9, 2023 at 21:53
  • $\begingroup$ I installed the version of the interface you indicated. Now, I get "R Session aborted" whenever I run knitro, even in the example problem HS40.R. Does the error also exist in the newer version 13.2.0? I'm asking because I cannot find any documentation on this. $\endgroup$
    – Frank
    Sep 9, 2023 at 22:16
  • $\begingroup$ Update: version 13.2.0 has the same bug. $\endgroup$
    – Frank
    Sep 9, 2023 at 22:54
  • $\begingroup$ Please contact the support, or contact me directly by email. That must be an installation issue $\endgroup$
    – fontanf
    Sep 10, 2023 at 7:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.