GA-Matlab


GA Matlab

Here is a simple example of using a GA in Matlab. The syntax is basically the same as with fmincon. Let's optimize the rosenbrock funciton.


In [24]:
function [f] = rosenbrock(x)

f = (1 - x(1))^2 + 100*(x(2) - x(1)^2)^2;
    
end

Here is where we set options. All are basically the same as fmincon, but instead of giving a starting point you give a number of variables. Why would that be? Recall that a GA starts with an inital population and not an initial starting point. Options are also different, and many exist. See documentation for details.


In [25]:
nvars = 2;
A = [];
b = [];
Aeq = [];
beq = [];
lb = [-5, -5];
ub = [5, 5];
con = [];
options = gaoptimset('Display', 'iter');

Now we can call ga.


In [26]:
[xopt, fopt, exitflag] = ga(@rosenbrock, nvars, A, b, Aeq, beq, lb, ub, con, options)


Best           Mean      Stall
Generation      f-count        f(x)           f(x)    Generations
    1           100           0.378            9569        0
    2           150          0.3538            6721        0
    3           200          0.3538            3588        1
    4           250          0.3538            2312        2
    5           300          0.3478            1380        0
    6           350          0.3478           84.66        1
    7           400          0.3478           56.41        2
    8           450          0.2301           38.47        0
    9           500          0.2301           33.75        1
   10           550          0.2301           24.33        2
   11           600          0.2301           24.55        3
   12           650          0.2239           22.12        0
   13           700          0.2239            21.8        1
   14           750          0.1936          0.6862        0
   15           800          0.1936           0.559        1
   16           850          0.1803          0.6689        0
   17           900          0.1803          0.5579        1
   18           950          0.1803          0.3605        2
   19          1000          0.1702          0.3373        0
   20          1050          0.1702          0.3245        1
   21          1100          0.1702          0.2719        2
   22          1150          0.1639          0.2153        0
   23          1200          0.1639          0.2219        1
   24          1250          0.1616          0.2085        0
   25          1300          0.1598          0.1787        0
   26          1350          0.1598          0.2087        1
   27          1400           0.157          0.1948        0
   28          1450          0.1558          0.2059        0
   29          1500          0.1216          0.2583        0
   30          1550          0.1216           0.736        1

                               Best           Mean      Stall
Generation      f-count        f(x)           f(x)    Generations
   31          1600          0.1216          0.4063        2
   32          1650          0.1176          0.4168        0
   33          1700          0.1176          0.4292        1
   34          1750          0.1176          0.3333        2
   35          1800          0.1096          0.2665        0
   36          1850          0.1025          0.3073        0
   37          1900          0.1025          0.3454        1
   38          1950          0.0987          0.2337        0
   39          2000          0.0987          0.2988        1
   40          2050          0.0987          0.2761        2
   41          2100         0.08852          0.2057        0
   42          2150         0.07663          0.1935        0
   43          2200         0.07663           0.306        1
   44          2250         0.07663          0.2837        2
   45          2300         0.07102          0.1815        0
   46          2350         0.06862          0.2222        0
   47          2400         0.06862          0.3892        1
   48          2450         0.06862          0.3496        2
   49          2500         0.06064          0.3131        0
   50          2550         0.05097          0.2393        0
   51          2600         0.05097          0.2646        1
   52          2650         0.05097          0.2772        2
   53          2700         0.05097          0.1932        3
   54          2750         0.04937          0.1604        0
   55          2800         0.04937          0.1233        1
   56          2850         0.04719          0.1025        0
   57          2900         0.04592          0.1042        0
   58          2950         0.04592         0.09972        1
   59          3000         0.04592         0.07503        2
   60          3050          0.0408         0.06704        0

                               Best           Mean      Stall
Generation      f-count        f(x)           f(x)    Generations
   61          3100          0.0408         0.06178        1
   62          3150          0.0408         0.06197        2
   63          3200          0.0408         0.05741        3
   64          3250          0.0408         0.06001        4
   65          3300         0.04004         0.05515        0
   66          3350         0.04004         0.04846        1
   67          3400         0.03953         0.04849        0
   68          3450         0.03949         0.04726        0
   69          3500         0.03949         0.04361        1
   70          3550         0.03876         0.04328        0
   71          3600         0.03867          0.0436        0
   72          3650         0.03767         0.04341        0
   73          3700         0.03767         0.05216        1
   74          3750         0.03767         0.05127        2
   75          3800         0.03767         0.04477        3
   76          3850         0.03719         0.04368        0
   77          3900         0.03701         0.04114        0
   78          3950         0.03668          0.0434        0
   79          4000         0.03668         0.05336        1
   80          4050         0.03536          0.0459        0
   81          4100         0.03536         0.05076        1
   82          4150         0.03536         0.04791        2
   83          4200         0.03474         0.04327        0
   84          4250         0.03426         0.04358        0
   85          4300         0.03407         0.05572        0
   86          4350         0.03407         0.09394        1
   87          4400         0.03084         0.07743        0
   88          4450         0.03084          0.1077        1
   89          4500         0.03084         0.08262        2
   90          4550         0.03084         0.07046        3

                               Best           Mean      Stall
Generation      f-count        f(x)           f(x)    Generations
   91          4600         0.03084         0.05947        4
   92          4650         0.03039           0.055        0
   93          4700         0.03029         0.04674        0
   94          4750         0.03029         0.03675        1
   95          4800         0.02976          0.0361        0
   96          4850         0.02934         0.03445        0
   97          4900         0.02934         0.04499        1
   98          4950         0.02835         0.03902        0
   99          5000         0.02835         0.04171        1
  100          5050         0.02803         0.04021        0
  101          5100         0.02803         0.04102        1
  102          5150         0.02679         0.03716        0
  103          5200         0.02679         0.04269        1
  104          5250         0.02679         0.03351        2
  105          5300         0.02679         0.03311        3
  106          5350         0.02679         0.03268        4
  107          5400         0.02679         0.02929        5
  108          5450         0.02669         0.02868        0
  109          5500          0.0266          0.0271        0
  110          5550         0.02642          0.0272        0
  111          5600         0.02642         0.02753        1
  112          5650         0.02618         0.02728        0
  113          5700         0.02618         0.02799        1
  114          5750         0.02618         0.02812        2
  115          5800         0.02582         0.02764        0
  116          5850         0.02582         0.02711        1
  117          5900         0.02582         0.02699        2
  118          5950         0.02562         0.02657        0
  119          6000         0.02537         0.02604        0
  120          6050         0.02522         0.02601        0

                               Best           Mean      Stall
Generation      f-count        f(x)           f(x)    Generations
  121          6100         0.02514         0.02639        0
  122          6150         0.02514         0.02978        1
  123          6200         0.02498         0.02843        0
  124          6250         0.02498         0.02965        1
  125          6300          0.0246         0.02756        0
  126          6350         0.02309         0.03077        0
  127          6400         0.02309          0.0372        1
  128          6450         0.02241         0.03787        0
  129          6500         0.02074         0.04634        0
  130          6550         0.02074         0.09201        1
  131          6600         0.02074         0.06039        2
  132          6650         0.01961         0.03897        0
  133          6700          0.0182         0.04046        0
  134          6750         0.01466         0.07519        0
  135          6800         0.00927          0.2623        0
  136          6850         0.00927          0.6887        1
  137          6900        0.004312          0.4405        0
  138          6950        0.004312           1.076        1
  139          7000        0.004312          0.9989        2
  140          7050        0.004312          0.9619        3
  141          7100        0.004312          0.6128        4
  142          7150        0.004312          0.3962        5
  143          7200        0.003522          0.1319        0
  144          7250        0.003463         0.07839        0
  145          7300        0.003463          0.0434        1
  146          7350        0.003463         0.01825        2
  147          7400        0.003463         0.01615        3
  148          7450         0.00339         0.01468        0
  149          7500         0.00339        0.006335        1
  150          7550         0.00339        0.005043        2

                               Best           Mean      Stall
Generation      f-count        f(x)           f(x)    Generations
  151          7600        0.003332        0.004208        0
  152          7650        0.003332         0.00373        1
  153          7700        0.003233        0.003617        0
  154          7750        0.003049        0.003571        0
  155          7800        0.003049        0.004835        1
  156          7850        0.003049        0.004315        2
  157          7900        0.003049        0.003776        3
  158          7950        0.003049        0.003625        4
  159          8000        0.003026        0.003441        0
  160          8050        0.003026        0.003457        1
  161          8100        0.003016        0.003343        0
  162          8150        0.003014        0.003316        0
  163          8200        0.002958        0.003127        0
  164          8250        0.002888        0.003356        0
  165          8300        0.002798        0.003954        0
  166          8350        0.002798        0.006122        1
  167          8400        0.002779        0.005492        0
  168          8450        0.002779        0.007086        1
  169          8500         0.00266        0.005324        0
  170          8550         0.00266        0.008267        1
  171          8600         0.00266        0.006004        2
  172          8650         0.00266        0.004905        3
  173          8700         0.00266        0.004137        4
  174          8750        0.002532        0.003379        0
  175          8800        0.002511        0.003163        0
  176          8850        0.002511        0.003281        1
  177          8900        0.002511         0.00302        2
  178          8950        0.002457        0.002969        0
  179          9000        0.002457        0.002935        1
  180          9050         0.00243        0.002822        0

                               Best           Mean      Stall
Generation      f-count        f(x)           f(x)    Generations
  181          9100         0.00243        0.002634        1
  182          9150        0.002386        0.002572        0
  183          9200        0.002386        0.002586        1
  184          9250        0.002386        0.002506        2
  185          9300        0.002386        0.002485        3
  186          9350        0.002384        0.002431        0
  187          9400        0.002382        0.002424        0
  188          9450        0.002354        0.002417        0
  189          9500        0.002354        0.002465        1
  190          9550        0.002338        0.002473        0
  191          9600        0.002338         0.00246        1
  192          9650        0.002338        0.002417        2
  193          9700        0.002328        0.002388        0
  194          9750        0.002313        0.002371        0
  195          9800        0.002306        0.002412        0
  196          9850        0.002306        0.002506        1
  197          9900        0.002292        0.002432        0
  198          9950        0.002177        0.002624        0
  199         10000        0.002155        0.003443        0
  200         10050        0.002155        0.004816        1
Optimization terminated: maximum number of generations exceeded.

xopt =

    0.9536    0.9096


fopt =

    0.0022


exitflag =

     0

Without changing any options, the default sets a max number of generations of 200 (100*nvars). You can increase the number of generations allowed by changing the option 'Generations'. I set it to 1000 and it still often reached the maximum iteration acount. The results are random so if you run it again it may be faster or slower with a different answer. This time we have pretty good answer, but convergence is no where near as tight as with a gradient-based method, and sometimes the answer is poor: $$x^* = (1, 1), f^* = 0$$ If it does converge it takes tens of thousands of function calls.

Let's also try with fmincon and start fairly far away (and I won't supply gradients):


In [27]:
x0 = [4.0, 4.0];
options = optimoptions(@fmincon, 'Display', 'iter');

In [28]:
[xopt, fopt, exitflag] = fmincon(rosen, x0, A, b, Aeq, beq, lb, ub, con, options)


First-order      Norm of
 Iter F-count            f(x)  Feasibility   optimality         step
    0       3    1.440900e+04    0.000e+00    1.909e+04
    1       7    1.430000e+03    0.000e+00    1.948e+03    4.478e+00
    2      10    7.201264e+02    0.000e+00    7.680e+02    7.876e-01
    3      15    4.539190e+01    0.000e+00    5.099e+02    1.189e+00
    4      19    8.025301e+00    0.000e+00    2.768e+01    2.099e-01
    5      22    7.020821e+00    0.000e+00    9.424e+00    2.527e-02
    6      25    6.992410e+00    0.000e+00    1.445e+00    5.450e-03
    7      28    6.986836e+00    0.000e+00    1.430e+00    3.775e-03
    8      31    6.954291e+00    0.000e+00    2.264e+00    2.149e-02
    9      34    6.788200e+00    0.000e+00    3.792e+00    1.087e-01
   10      37    6.077078e+00    0.000e+00    2.264e+01    5.429e-01
   11      41    5.938470e+00    0.000e+00    4.674e+01    4.807e-01
   12      44    5.500198e+00    0.000e+00    2.416e+01    9.286e-02
   13      47    4.945856e+00    0.000e+00    1.043e+01    2.366e-01
   14      51    4.425827e+00    0.000e+00    2.158e+01    4.183e-01
   15      54    4.164888e+00    0.000e+00    3.040e+01    3.819e-01
   16      57    3.517213e+00    0.000e+00    1.070e+01    4.202e-02
   17      60    2.999341e+00    0.000e+00    1.467e+01    3.551e-01
   18      63    2.590856e+00    0.000e+00    1.228e+01    1.995e-01
   19      66    2.167985e+00    0.000e+00    5.682e+00    1.189e-01
   20      70    1.781976e+00    0.000e+00    1.068e+01    3.190e-01
   21      73    1.713915e+00    0.000e+00    2.330e+00    1.326e-01
   22      76    1.538536e+00    0.000e+00    3.596e+00    9.238e-02
   23      80    1.460693e+00    0.000e+00    9.958e+00    1.568e-01
   24      83    1.332303e+00    0.000e+00    4.145e+00    5.054e-02
   25      86    1.203740e+00    0.000e+00    2.572e+00    4.338e-02
   26      89    1.003363e+00    0.000e+00    9.644e+00    2.164e-01
   27      92    8.707986e-01    0.000e+00    1.763e+00    6.427e-02
   28      95    7.432283e-01    0.000e+00    1.497e+00    7.447e-02
   29      99    6.953287e-01    0.000e+00    9.367e+00    1.729e-01
   30     102    5.672629e-01    0.000e+00    1.171e+00    6.138e-02

                                            First-order      Norm of
 Iter F-count            f(x)  Feasibility   optimality         step
   31     105    4.887385e-01    0.000e+00    1.249e+00    6.218e-02
   32     109    3.937533e-01    0.000e+00    4.732e+00    1.326e-01
   33     112    3.157171e-01    0.000e+00    3.154e+00    6.129e-02
   34     115    2.178491e-01    0.000e+00    3.217e+00    1.445e-01
   35     118    1.960288e-01    0.000e+00    6.828e+00    1.381e-01
   36     122    6.831299e-02    0.000e+00    8.284e-01    1.692e-01
   37     126    6.674511e-02    0.000e+00    3.908e+00    6.512e-02
   38     129    5.531226e-02    0.000e+00    1.724e+00    7.417e-03
   39     132    4.146920e-02    0.000e+00    3.626e-01    4.498e-02
   40     135    1.950753e-02    0.000e+00    8.601e-01    1.287e-01
   41     138    1.480825e-02    0.000e+00    3.632e+00    1.335e-01
   42     141    7.233980e-03    0.000e+00    4.755e-01    2.503e-02
   43     144    3.962467e-03    0.000e+00    1.436e-01    4.454e-02
   44     147    1.088291e-03    0.000e+00    8.926e-01    8.438e-02
   45     150    5.188312e-04    0.000e+00    9.927e-02    2.639e-03
   46     153    2.116570e-04    0.000e+00    2.379e-02    1.783e-02
   47     156    1.585994e-05    0.000e+00    8.755e-02    2.491e-02
   48     159    7.576341e-06    0.000e+00    4.997e-03    1.295e-03
   49     162    1.293592e-08    0.000e+00    4.569e-03    6.135e-03
   50     165    1.056943e-09    0.000e+00    5.266e-05    5.543e-05
   51     168    2.622776e-11    0.000e+00    2.878e-06    6.121e-05
   52     171    2.235886e-11    0.000e+00    5.000e-07    8.815e-07

Local minimum found that satisfies the constraints.

Optimization completed because the objective function is non-decreasing in 
feasible directions, to within the default value of the function tolerance,
and constraints are satisfied to within the default value of the constraint tolerance.




xopt =

    1.0000    1.0000


fopt =

   2.2359e-11


exitflag =

     1

We have the answer to high precision, and its fast and repeatable. For something that is differentiable a gradient-based method is preferable, but if the function space is fundamentally noisy, discrete, or highly multi-modal then a GA or other gradient-free method can be effective.


In [ ]: