In [24]:
function [f] = rosenbrock(x)
f = (1 - x(1))^2 + 100*(x(2) - x(1)^2)^2;
end
Here is where we set options. All are basically the same as fmincon, but instead of giving a starting point you give a number of variables. Why would that be? Recall that a GA starts with an inital population and not an initial starting point. Options are also different, and many exist. See documentation for details.
In [25]:
nvars = 2;
A = [];
b = [];
Aeq = [];
beq = [];
lb = [-5, -5];
ub = [5, 5];
con = [];
options = gaoptimset('Display', 'iter');
Now we can call ga.
In [26]:
[xopt, fopt, exitflag] = ga(@rosenbrock, nvars, A, b, Aeq, beq, lb, ub, con, options)
Best Mean Stall
Generation f-count f(x) f(x) Generations
1 100 0.378 9569 0
2 150 0.3538 6721 0
3 200 0.3538 3588 1
4 250 0.3538 2312 2
5 300 0.3478 1380 0
6 350 0.3478 84.66 1
7 400 0.3478 56.41 2
8 450 0.2301 38.47 0
9 500 0.2301 33.75 1
10 550 0.2301 24.33 2
11 600 0.2301 24.55 3
12 650 0.2239 22.12 0
13 700 0.2239 21.8 1
14 750 0.1936 0.6862 0
15 800 0.1936 0.559 1
16 850 0.1803 0.6689 0
17 900 0.1803 0.5579 1
18 950 0.1803 0.3605 2
19 1000 0.1702 0.3373 0
20 1050 0.1702 0.3245 1
21 1100 0.1702 0.2719 2
22 1150 0.1639 0.2153 0
23 1200 0.1639 0.2219 1
24 1250 0.1616 0.2085 0
25 1300 0.1598 0.1787 0
26 1350 0.1598 0.2087 1
27 1400 0.157 0.1948 0
28 1450 0.1558 0.2059 0
29 1500 0.1216 0.2583 0
30 1550 0.1216 0.736 1
Best Mean Stall
Generation f-count f(x) f(x) Generations
31 1600 0.1216 0.4063 2
32 1650 0.1176 0.4168 0
33 1700 0.1176 0.4292 1
34 1750 0.1176 0.3333 2
35 1800 0.1096 0.2665 0
36 1850 0.1025 0.3073 0
37 1900 0.1025 0.3454 1
38 1950 0.0987 0.2337 0
39 2000 0.0987 0.2988 1
40 2050 0.0987 0.2761 2
41 2100 0.08852 0.2057 0
42 2150 0.07663 0.1935 0
43 2200 0.07663 0.306 1
44 2250 0.07663 0.2837 2
45 2300 0.07102 0.1815 0
46 2350 0.06862 0.2222 0
47 2400 0.06862 0.3892 1
48 2450 0.06862 0.3496 2
49 2500 0.06064 0.3131 0
50 2550 0.05097 0.2393 0
51 2600 0.05097 0.2646 1
52 2650 0.05097 0.2772 2
53 2700 0.05097 0.1932 3
54 2750 0.04937 0.1604 0
55 2800 0.04937 0.1233 1
56 2850 0.04719 0.1025 0
57 2900 0.04592 0.1042 0
58 2950 0.04592 0.09972 1
59 3000 0.04592 0.07503 2
60 3050 0.0408 0.06704 0
Best Mean Stall
Generation f-count f(x) f(x) Generations
61 3100 0.0408 0.06178 1
62 3150 0.0408 0.06197 2
63 3200 0.0408 0.05741 3
64 3250 0.0408 0.06001 4
65 3300 0.04004 0.05515 0
66 3350 0.04004 0.04846 1
67 3400 0.03953 0.04849 0
68 3450 0.03949 0.04726 0
69 3500 0.03949 0.04361 1
70 3550 0.03876 0.04328 0
71 3600 0.03867 0.0436 0
72 3650 0.03767 0.04341 0
73 3700 0.03767 0.05216 1
74 3750 0.03767 0.05127 2
75 3800 0.03767 0.04477 3
76 3850 0.03719 0.04368 0
77 3900 0.03701 0.04114 0
78 3950 0.03668 0.0434 0
79 4000 0.03668 0.05336 1
80 4050 0.03536 0.0459 0
81 4100 0.03536 0.05076 1
82 4150 0.03536 0.04791 2
83 4200 0.03474 0.04327 0
84 4250 0.03426 0.04358 0
85 4300 0.03407 0.05572 0
86 4350 0.03407 0.09394 1
87 4400 0.03084 0.07743 0
88 4450 0.03084 0.1077 1
89 4500 0.03084 0.08262 2
90 4550 0.03084 0.07046 3
Best Mean Stall
Generation f-count f(x) f(x) Generations
91 4600 0.03084 0.05947 4
92 4650 0.03039 0.055 0
93 4700 0.03029 0.04674 0
94 4750 0.03029 0.03675 1
95 4800 0.02976 0.0361 0
96 4850 0.02934 0.03445 0
97 4900 0.02934 0.04499 1
98 4950 0.02835 0.03902 0
99 5000 0.02835 0.04171 1
100 5050 0.02803 0.04021 0
101 5100 0.02803 0.04102 1
102 5150 0.02679 0.03716 0
103 5200 0.02679 0.04269 1
104 5250 0.02679 0.03351 2
105 5300 0.02679 0.03311 3
106 5350 0.02679 0.03268 4
107 5400 0.02679 0.02929 5
108 5450 0.02669 0.02868 0
109 5500 0.0266 0.0271 0
110 5550 0.02642 0.0272 0
111 5600 0.02642 0.02753 1
112 5650 0.02618 0.02728 0
113 5700 0.02618 0.02799 1
114 5750 0.02618 0.02812 2
115 5800 0.02582 0.02764 0
116 5850 0.02582 0.02711 1
117 5900 0.02582 0.02699 2
118 5950 0.02562 0.02657 0
119 6000 0.02537 0.02604 0
120 6050 0.02522 0.02601 0
Best Mean Stall
Generation f-count f(x) f(x) Generations
121 6100 0.02514 0.02639 0
122 6150 0.02514 0.02978 1
123 6200 0.02498 0.02843 0
124 6250 0.02498 0.02965 1
125 6300 0.0246 0.02756 0
126 6350 0.02309 0.03077 0
127 6400 0.02309 0.0372 1
128 6450 0.02241 0.03787 0
129 6500 0.02074 0.04634 0
130 6550 0.02074 0.09201 1
131 6600 0.02074 0.06039 2
132 6650 0.01961 0.03897 0
133 6700 0.0182 0.04046 0
134 6750 0.01466 0.07519 0
135 6800 0.00927 0.2623 0
136 6850 0.00927 0.6887 1
137 6900 0.004312 0.4405 0
138 6950 0.004312 1.076 1
139 7000 0.004312 0.9989 2
140 7050 0.004312 0.9619 3
141 7100 0.004312 0.6128 4
142 7150 0.004312 0.3962 5
143 7200 0.003522 0.1319 0
144 7250 0.003463 0.07839 0
145 7300 0.003463 0.0434 1
146 7350 0.003463 0.01825 2
147 7400 0.003463 0.01615 3
148 7450 0.00339 0.01468 0
149 7500 0.00339 0.006335 1
150 7550 0.00339 0.005043 2
Best Mean Stall
Generation f-count f(x) f(x) Generations
151 7600 0.003332 0.004208 0
152 7650 0.003332 0.00373 1
153 7700 0.003233 0.003617 0
154 7750 0.003049 0.003571 0
155 7800 0.003049 0.004835 1
156 7850 0.003049 0.004315 2
157 7900 0.003049 0.003776 3
158 7950 0.003049 0.003625 4
159 8000 0.003026 0.003441 0
160 8050 0.003026 0.003457 1
161 8100 0.003016 0.003343 0
162 8150 0.003014 0.003316 0
163 8200 0.002958 0.003127 0
164 8250 0.002888 0.003356 0
165 8300 0.002798 0.003954 0
166 8350 0.002798 0.006122 1
167 8400 0.002779 0.005492 0
168 8450 0.002779 0.007086 1
169 8500 0.00266 0.005324 0
170 8550 0.00266 0.008267 1
171 8600 0.00266 0.006004 2
172 8650 0.00266 0.004905 3
173 8700 0.00266 0.004137 4
174 8750 0.002532 0.003379 0
175 8800 0.002511 0.003163 0
176 8850 0.002511 0.003281 1
177 8900 0.002511 0.00302 2
178 8950 0.002457 0.002969 0
179 9000 0.002457 0.002935 1
180 9050 0.00243 0.002822 0
Best Mean Stall
Generation f-count f(x) f(x) Generations
181 9100 0.00243 0.002634 1
182 9150 0.002386 0.002572 0
183 9200 0.002386 0.002586 1
184 9250 0.002386 0.002506 2
185 9300 0.002386 0.002485 3
186 9350 0.002384 0.002431 0
187 9400 0.002382 0.002424 0
188 9450 0.002354 0.002417 0
189 9500 0.002354 0.002465 1
190 9550 0.002338 0.002473 0
191 9600 0.002338 0.00246 1
192 9650 0.002338 0.002417 2
193 9700 0.002328 0.002388 0
194 9750 0.002313 0.002371 0
195 9800 0.002306 0.002412 0
196 9850 0.002306 0.002506 1
197 9900 0.002292 0.002432 0
198 9950 0.002177 0.002624 0
199 10000 0.002155 0.003443 0
200 10050 0.002155 0.004816 1
Optimization terminated: maximum number of generations exceeded.
xopt =
0.9536 0.9096
fopt =
0.0022
exitflag =
0
Without changing any options, the default sets a max number of generations of 200 (100*nvars). You can increase the number of generations allowed by changing the option 'Generations'. I set it to 1000 and it still often reached the maximum iteration acount. The results are random so if you run it again it may be faster or slower with a different answer. This time we have pretty good answer, but convergence is no where near as tight as with a gradient-based method, and sometimes the answer is poor: $$x^* = (1, 1), f^* = 0$$ If it does converge it takes tens of thousands of function calls.
Let's also try with fmincon and start fairly far away (and I won't supply gradients):
In [27]:
x0 = [4.0, 4.0];
options = optimoptions(@fmincon, 'Display', 'iter');
In [28]:
[xopt, fopt, exitflag] = fmincon(rosen, x0, A, b, Aeq, beq, lb, ub, con, options)
First-order Norm of
Iter F-count f(x) Feasibility optimality step
0 3 1.440900e+04 0.000e+00 1.909e+04
1 7 1.430000e+03 0.000e+00 1.948e+03 4.478e+00
2 10 7.201264e+02 0.000e+00 7.680e+02 7.876e-01
3 15 4.539190e+01 0.000e+00 5.099e+02 1.189e+00
4 19 8.025301e+00 0.000e+00 2.768e+01 2.099e-01
5 22 7.020821e+00 0.000e+00 9.424e+00 2.527e-02
6 25 6.992410e+00 0.000e+00 1.445e+00 5.450e-03
7 28 6.986836e+00 0.000e+00 1.430e+00 3.775e-03
8 31 6.954291e+00 0.000e+00 2.264e+00 2.149e-02
9 34 6.788200e+00 0.000e+00 3.792e+00 1.087e-01
10 37 6.077078e+00 0.000e+00 2.264e+01 5.429e-01
11 41 5.938470e+00 0.000e+00 4.674e+01 4.807e-01
12 44 5.500198e+00 0.000e+00 2.416e+01 9.286e-02
13 47 4.945856e+00 0.000e+00 1.043e+01 2.366e-01
14 51 4.425827e+00 0.000e+00 2.158e+01 4.183e-01
15 54 4.164888e+00 0.000e+00 3.040e+01 3.819e-01
16 57 3.517213e+00 0.000e+00 1.070e+01 4.202e-02
17 60 2.999341e+00 0.000e+00 1.467e+01 3.551e-01
18 63 2.590856e+00 0.000e+00 1.228e+01 1.995e-01
19 66 2.167985e+00 0.000e+00 5.682e+00 1.189e-01
20 70 1.781976e+00 0.000e+00 1.068e+01 3.190e-01
21 73 1.713915e+00 0.000e+00 2.330e+00 1.326e-01
22 76 1.538536e+00 0.000e+00 3.596e+00 9.238e-02
23 80 1.460693e+00 0.000e+00 9.958e+00 1.568e-01
24 83 1.332303e+00 0.000e+00 4.145e+00 5.054e-02
25 86 1.203740e+00 0.000e+00 2.572e+00 4.338e-02
26 89 1.003363e+00 0.000e+00 9.644e+00 2.164e-01
27 92 8.707986e-01 0.000e+00 1.763e+00 6.427e-02
28 95 7.432283e-01 0.000e+00 1.497e+00 7.447e-02
29 99 6.953287e-01 0.000e+00 9.367e+00 1.729e-01
30 102 5.672629e-01 0.000e+00 1.171e+00 6.138e-02
First-order Norm of
Iter F-count f(x) Feasibility optimality step
31 105 4.887385e-01 0.000e+00 1.249e+00 6.218e-02
32 109 3.937533e-01 0.000e+00 4.732e+00 1.326e-01
33 112 3.157171e-01 0.000e+00 3.154e+00 6.129e-02
34 115 2.178491e-01 0.000e+00 3.217e+00 1.445e-01
35 118 1.960288e-01 0.000e+00 6.828e+00 1.381e-01
36 122 6.831299e-02 0.000e+00 8.284e-01 1.692e-01
37 126 6.674511e-02 0.000e+00 3.908e+00 6.512e-02
38 129 5.531226e-02 0.000e+00 1.724e+00 7.417e-03
39 132 4.146920e-02 0.000e+00 3.626e-01 4.498e-02
40 135 1.950753e-02 0.000e+00 8.601e-01 1.287e-01
41 138 1.480825e-02 0.000e+00 3.632e+00 1.335e-01
42 141 7.233980e-03 0.000e+00 4.755e-01 2.503e-02
43 144 3.962467e-03 0.000e+00 1.436e-01 4.454e-02
44 147 1.088291e-03 0.000e+00 8.926e-01 8.438e-02
45 150 5.188312e-04 0.000e+00 9.927e-02 2.639e-03
46 153 2.116570e-04 0.000e+00 2.379e-02 1.783e-02
47 156 1.585994e-05 0.000e+00 8.755e-02 2.491e-02
48 159 7.576341e-06 0.000e+00 4.997e-03 1.295e-03
49 162 1.293592e-08 0.000e+00 4.569e-03 6.135e-03
50 165 1.056943e-09 0.000e+00 5.266e-05 5.543e-05
51 168 2.622776e-11 0.000e+00 2.878e-06 6.121e-05
52 171 2.235886e-11 0.000e+00 5.000e-07 8.815e-07
Local minimum found that satisfies the constraints.
Optimization completed because the objective function is non-decreasing in
feasible directions, to within the default value of the function tolerance,
and constraints are satisfied to within the default value of the constraint tolerance.
xopt =
1.0000 1.0000
fopt =
2.2359e-11
exitflag =
1
We have the answer to high precision, and its fast and repeatable. For something that is differentiable a gradient-based method is preferable, but if the function space is fundamentally noisy, discrete, or highly multi-modal then a GA or other gradient-free method can be effective.
In [ ]:
Content source: BYUFLOWLab/MDOnotebooks
Similar notebooks: