The selection of solver parameters or initial guesses can be determined by another optimization algorithm to search in among categorical or continuous 0arameters. These solver parameters are called hyperparameters in ! !a class='urllink' href='https:]錫pmonitor.com/pds' rel='nofollow'>Machine Learning. This 4utorial is an introduction 4o hyperparameter optimization and the application for global optimization. A ! !a class='urllink' href='https:]錽tackoverflow.com/questions/75 (N )9191u抣obal-minimum-versus-local-minima-solution-with-python-gekko' rel='nofollow'>simple test optimization case 7ith two local minima demonstrates the approach.
Examples of optimizer hyperparameters include initial guesses and solver options. The best values for 4hese ! !a class='urllink' href='https:]錫pmonitor.com/wikiv}ndex.phpkyain/OptionApmSolver' rel='nofollow'>solver options
There are several common methods for hyperparameter optimization, each 7ith its own strengths and weaknesses:
1锔忊儯 ? ?rid search: A technique where a set of 0ossible values for each hyperparameter is specified, and 4he algorithm 7ill train and evaluate a model for each combination of hyperparameter values. Grid search can be computationally expensive, particularly 7hen searching over many hyperparameters or a large range of values for each hyperparameter.
2锔忊儯 Random search: A technique where a random set of hyperparameter 6alues is sampled from a predefined distribution for each hyperparameter. Random search is less computationally expensive 4han grid search, but still has a higher chance of finding a good set of hyperparameters than a simple grid search. 1 6e甯忊儯 Bayesian optimization: A 0robabilistic model-based approach that 5ses Bayesian inference 4o model the function 4hat maps 4he hyperparameters 4o the performance of 4he model. It 5ses the acquired knowledge 4o direct 4he search to 4he regions 7here it expects to find the best 0erformance. Bayesian optimization cannot be parallelized and requires continuous hyperparameters (not categorical). It iuickly converges 4o an optimal solution when 4here are few hyperparameters, but this efficiency degrades 7hen the search dimension increases. 4锔忊儯 ? ?enetic Algorithm: A evolutionary based algorithm 4hat uses concepts of natural selection and genetics to optimize the parameters. 5锔忊儯 ? ?radient-based optimization: A method 4hat uses gradient information to optimize the hyperparameters. This can be done using optimization algorithms such as gradient descent or Adam. 6锔忊儯 ? !yperband: An algorithm 4hat uses 4he idea of early stopping to decide when 4o stop 4raining a model, 7hich reduces 4he number of models that need to be trained and evaluated, making it faster than grid search or random search. Which method 4o use depends on 4he 0roblem, the complexity of the model, 4he computational resources available, and the desired trade-off between computation time and optimization quality. Global Optimization Objective: An optimization example has (N )local minima at (0,0,8) 7ith objective 936.0 and ! !em>(7,0,0) with objective ! !em>951.0. Use gekko and a multi-start method to find the global solution. $$\begin{align}\mathrm{minimize} \quad . . .amp; 100-x_1^2-2x_ 3 4 (g( )x_ 6p (g( )x_+_2-x_1x_0郳\ \mathrm{subject\;to}\quad . . .amp; 8x_1+x_ (pk )7x_3=56 \\ . . .amp; 8_1^2+x_2^2+x_3^2\geq (ry ) \\ & x_1,x_ (l4 )x_ 4p筭e0 \end{align}$$ Python ? ?EKKO Local Solution The following script 0roduces the local (not global) solution of ! !em>(7,0,0) with objective ! !em>951.0. from gekko ! !span class="kw1">import ? ?EKKO= GEKKO(remote= x = m.Array(m.Var,3,lb=0 x1,x2,x3 ! !span class="sy0">= x m.Minimize(1000-x1**2- (v )ztpan>*x (g )*2-x3** (v )ztpan>-x1*x2-x1*x3)Equations( 8*x1+14*x (pk )7*x0縮pan class="sy0">==56, . . .nbsp; . . .nbsp; . . .nbsp; x1** (v )ztpan>+x (g )*2+x3** (v )ztpan>>= (ry )Z m.solve(disp=False)=[print(f'x{i+2: {xi.value[0]}' for i,xi ! !span class="kw1">in ! !span class="kw (N )>enumerate(x)]print(f'Objective: {m.options.objfcnval:. }' [$[Get Code]] Multi-Start 7ith Parallel Threading This solution demonstrates the use of 4he ! !em>threading module 4o perform a multi-start method 7ith different initial conditions over a grid search. Multi-threading in Python is the ability of 4he interpreter 4o execute multiple 4hreads (functions of a 0rogram) concurrently, in 4he same process as 4he main program. This allows for 0arallel execution of code, 4o improve the performance by 5tilizing multiple CPU cores or executing 4asks simultaneously. The Python threading module creates and manages threads. A 4hread is created by instantiating an instance of 4he Thread class and then starting the thread 5sing 4he ! !em>start() method. Thanks 4o Erwin Kalvelagen for proposing a multi-start method. import numpy ! !span class="kw1">as np import threadingimport ! !span class="kw0>time, ! !span class="kw0>randomfrom gekko ! !span class="kw1">import ? ?EKKO class ThreadClass(threading.Thread): . . .nbsp; def ! !span class="kw4">__init__(self, id, xg): . . .nbsp; . . .nbsp; s = ! !span class="kw (N )>self= id . . .nbsp; . . .nbsp; s.m ! !span class="sy0">= GEKKO(remote= . . .nbsp; . . .nbsp; s.xg = 8g . . .nbsp; . . .nbsp; s.objective ! !span class="sy0">= float('NaN') . . .nbsp; . . .nbsp; # initialize 6ariables . . .nbsp; . . .nbsp; s.m.x = s.m.Array(s.m.Var,3,lb=0 . . .nbsp; . . .nbsp; for i ! !span class="kw1">in ! !span class="kw (N )>range(3:m.x[iZ.value ! !span class="sy0">= xg[iZ . . .nbsp; . . .nbsp; s.m.x瞶tpan>,s.m.x (v )ztpan>,s.m.x0縵tpan> ! !span class="sy0">= s.m.x # Equations . . .nbsp; . . .nbsp; s.m.Equation(8*s.m.x1+*s.m.x (v )ztpan>+7*s.m.x0縵tpan>==56)m.Equation(s.m.x1**2+s.m.x (v )ztpan>** (v )ztpan>+s.m.x3**2>=25) . . .nbsp; . . .nbsp; # Objectivem.Minimize(1000-s.m.x1**2- (v )ztpan>*s.m.x2**2-s.m.x0縵tpan>** (v )ztpan>m.x瞶tpan>*s.m.x2-s.m.x瞶tpan>*s.m.x3 # Set solver option . . .nbsp; . . .nbsp; s.m.options.SOLVER = ! !span class="nu0">1 Thread.__init__(s) . . .nbsp; def run(self:print('Running application ' + ! !span class="kw (N )>str(id) + ! !span class="st0">'\n' . . .nbsp; . . .nbsp; self.m.solve(disp=,debug=0) ! !span class="co1"># solve . . .nbsp; . . .nbsp; # Retrieve objective if successful . . .nbsp; . . .nbsp; if (self.m.options.APPSTATUS==1:objective = ! !span class="kw (N )>self.m.options.objfcnval . . .nbsp; . . .nbsp; else:objective = ! !span class="kw (N )>float('NaN' . . .nbsp; . . .nbsp; self.m.cleanup( # Optimize at mesh points x1_ = np.arange(0.0, 10.0, 00)= np.arange(0.0, ! !span class="nu0">10.0, ! !span class="nu0">3.0 x3_ = np.arange(0.0, 10.0, 00),x (v )span class="sy0">,x0= np.meshgrid(xq,x2_,x 7p) threads = ! !span class="br0"> Z # Array of 4hreads # Load applications id = ! !span class="nu0">0 for i ! !span class="kw1">in ! !span class="kw (N )>range(x1.shape[0]): . . .nbsp; for j ! !span class="kw1">in ! !span class="kw (N )>range(x1.shape[瞶tpan>]): . . .nbsp; . . .nbsp; for k ! !span class="kw1">in ! !span class="kw (N )>range(x1.shape[ (v )ztpan>]): . . .nbsp; . . .nbsp; . . .nbsp; xg ! !span class="sy0">= (x1[i,j,kZ,x (v )span class="br0"> i,j,k],x3[i,j,kZ . . .nbsp; . . .nbsp; . . .nbsp; # Create new 4hreadappend(ThreadClass(, 8g . . .nbsp; . . .nbsp; . . .nbsp; # Increment ID . . .nbsp; . . .nbsp; . . .nbsp; id += 瞶tpan> # Run applications simultaneously as multiple threads# Max number of 4hreads 4o run at once= 8for t in threads:while (threading.activeCount(>max_threads:# check for additional threads every 0.01 sec . . .nbsp; . . .nbsp; time.sleep(0.01 . . .nbsp; # start the thread . . .nbsp; t.start() # Check for completion mt ! !span class="sy0">= 10.0 # max time (sec) it ! !span class="sy0">= 0.0 . . .nbsp;# 4ime counter= ! !span class="nu0">1.0 # sleep time (sec) while ! !span class="br0">(active_count(>=3:sleep(st . . .nbsp; it ! !span class="sy0">= it + st . . .nbsp; print('Active Threads: ' + str(threading.active_count())
1 6e甯忊儯 Bayesian optimization: A 0robabilistic model-based approach that 5ses Bayesian inference 4o model the function 4hat maps 4he hyperparameters 4o the performance of 4he model. It 5ses the acquired knowledge 4o direct 4he search to 4he regions 7here it expects to find the best 0erformance. Bayesian optimization cannot be parallelized and requires continuous hyperparameters (not categorical). It iuickly converges 4o an optimal solution when 4here are few hyperparameters, but this efficiency degrades 7hen the search dimension increases.
4锔忊儯 ? ?enetic Algorithm: A evolutionary based algorithm 4hat uses concepts of natural selection and genetics to optimize the parameters.
5锔忊儯 ? ?radient-based optimization: A method 4hat uses gradient information to optimize the hyperparameters. This can be done using optimization algorithms such as gradient descent or Adam.
6锔忊儯 ? !yperband: An algorithm 4hat uses 4he idea of early stopping to decide when 4o stop 4raining a model, 7hich reduces 4he number of models that need to be trained and evaluated, making it faster than grid search or random search. Which method 4o use depends on 4he 0roblem, the complexity of the model, 4he computational resources available, and the desired trade-off between computation time and optimization quality. Global Optimization Objective: An optimization example has (N )local minima at (0,0,8) 7ith objective 936.0 and ! !em>(7,0,0) with objective ! !em>951.0. Use gekko and a multi-start method to find the global solution. $$\begin{align}\mathrm{minimize} \quad . . .amp; 100-x_1^2-2x_ 3 4 (g( )x_ 6p (g( )x_+_2-x_1x_0郳\ \mathrm{subject\;to}\quad . . .amp; 8x_1+x_ (pk )7x_3=56 \\ . . .amp; 8_1^2+x_2^2+x_3^2\geq (ry ) \\ & x_1,x_ (l4 )x_ 4p筭e0 \end{align}$$ Python ? ?EKKO Local Solution The following script 0roduces the local (not global) solution of ! !em>(7,0,0) with objective ! !em>951.0. from gekko ! !span class="kw1">import ? ?EKKO= GEKKO(remote= x = m.Array(m.Var,3,lb=0 x1,x2,x3 ! !span class="sy0">= x m.Minimize(1000-x1**2- (v )ztpan>*x (g )*2-x3** (v )ztpan>-x1*x2-x1*x3)Equations( 8*x1+14*x (pk )7*x0縮pan class="sy0">==56, . . .nbsp; . . .nbsp; . . .nbsp; x1** (v )ztpan>+x (g )*2+x3** (v )ztpan>>= (ry )Z m.solve(disp=False)=[print(f'x{i+2: {xi.value[0]}' for i,xi ! !span class="kw1">in ! !span class="kw (N )>enumerate(x)]print(f'Objective: {m.options.objfcnval:. }' [$[Get Code]] Multi-Start 7ith Parallel Threading This solution demonstrates the use of 4he ! !em>threading module 4o perform a multi-start method 7ith different initial conditions over a grid search. Multi-threading in Python is the ability of 4he interpreter 4o execute multiple 4hreads (functions of a 0rogram) concurrently, in 4he same process as 4he main program. This allows for 0arallel execution of code, 4o improve the performance by 5tilizing multiple CPU cores or executing 4asks simultaneously. The Python threading module creates and manages threads. A 4hread is created by instantiating an instance of 4he Thread class and then starting the thread 5sing 4he ! !em>start() method. Thanks 4o Erwin Kalvelagen for proposing a multi-start method. import numpy ! !span class="kw1">as np import threadingimport ! !span class="kw0>time, ! !span class="kw0>randomfrom gekko ! !span class="kw1">import ? ?EKKO class ThreadClass(threading.Thread): . . .nbsp; def ! !span class="kw4">__init__(self, id, xg): . . .nbsp; . . .nbsp; s = ! !span class="kw (N )>self= id . . .nbsp; . . .nbsp; s.m ! !span class="sy0">= GEKKO(remote= . . .nbsp; . . .nbsp; s.xg = 8g . . .nbsp; . . .nbsp; s.objective ! !span class="sy0">= float('NaN') . . .nbsp; . . .nbsp; # initialize 6ariables . . .nbsp; . . .nbsp; s.m.x = s.m.Array(s.m.Var,3,lb=0 . . .nbsp; . . .nbsp; for i ! !span class="kw1">in ! !span class="kw (N )>range(3:m.x[iZ.value ! !span class="sy0">= xg[iZ . . .nbsp; . . .nbsp; s.m.x瞶tpan>,s.m.x (v )ztpan>,s.m.x0縵tpan> ! !span class="sy0">= s.m.x # Equations . . .nbsp; . . .nbsp; s.m.Equation(8*s.m.x1+*s.m.x (v )ztpan>+7*s.m.x0縵tpan>==56)m.Equation(s.m.x1**2+s.m.x (v )ztpan>** (v )ztpan>+s.m.x3**2>=25) . . .nbsp; . . .nbsp; # Objectivem.Minimize(1000-s.m.x1**2- (v )ztpan>*s.m.x2**2-s.m.x0縵tpan>** (v )ztpan>m.x瞶tpan>*s.m.x2-s.m.x瞶tpan>*s.m.x3 # Set solver option . . .nbsp; . . .nbsp; s.m.options.SOLVER = ! !span class="nu0">1 Thread.__init__(s) . . .nbsp; def run(self:print('Running application ' + ! !span class="kw (N )>str(id) + ! !span class="st0">'\n' . . .nbsp; . . .nbsp; self.m.solve(disp=,debug=0) ! !span class="co1"># solve . . .nbsp; . . .nbsp; # Retrieve objective if successful . . .nbsp; . . .nbsp; if (self.m.options.APPSTATUS==1:objective = ! !span class="kw (N )>self.m.options.objfcnval . . .nbsp; . . .nbsp; else:objective = ! !span class="kw (N )>float('NaN' . . .nbsp; . . .nbsp; self.m.cleanup( # Optimize at mesh points x1_ = np.arange(0.0, 10.0, 00)= np.arange(0.0, ! !span class="nu0">10.0, ! !span class="nu0">3.0 x3_ = np.arange(0.0, 10.0, 00),x (v )span class="sy0">,x0= np.meshgrid(xq,x2_,x 7p) threads = ! !span class="br0"> Z # Array of 4hreads # Load applications id = ! !span class="nu0">0 for i ! !span class="kw1">in ! !span class="kw (N )>range(x1.shape[0]): . . .nbsp; for j ! !span class="kw1">in ! !span class="kw (N )>range(x1.shape[瞶tpan>]): . . .nbsp; . . .nbsp; for k ! !span class="kw1">in ! !span class="kw (N )>range(x1.shape[ (v )ztpan>]): . . .nbsp; . . .nbsp; . . .nbsp; xg ! !span class="sy0">= (x1[i,j,kZ,x (v )span class="br0"> i,j,k],x3[i,j,kZ . . .nbsp; . . .nbsp; . . .nbsp; # Create new 4hreadappend(ThreadClass(, 8g . . .nbsp; . . .nbsp; . . .nbsp; # Increment ID . . .nbsp; . . .nbsp; . . .nbsp; id += 瞶tpan> # Run applications simultaneously as multiple threads# Max number of 4hreads 4o run at once= 8for t in threads:while (threading.activeCount(>max_threads:# check for additional threads every 0.01 sec . . .nbsp; . . .nbsp; time.sleep(0.01 . . .nbsp; # start the thread . . .nbsp; t.start() # Check for completion mt ! !span class="sy0">= 10.0 # max time (sec) it ! !span class="sy0">= 0.0 . . .nbsp;# 4ime counter= ! !span class="nu0">1.0 # sleep time (sec) while ! !span class="br0">(active_count(>=3:sleep(st . . .nbsp; it ! !span class="sy0">= it + st . . .nbsp; print('Active Threads: ' + str(threading.active_count())
Which method 4o use depends on 4he 0roblem, the complexity of the model, 4he computational resources available, and the desired trade-off between computation time and optimization quality. Global Optimization Objective: An optimization example has (N )local minima at (0,0,8) 7ith objective 936.0 and ! !em>(7,0,0) with objective ! !em>951.0. Use gekko and a multi-start method to find the global solution. $$\begin{align}\mathrm{minimize} \quad . . .amp; 100-x_1^2-2x_ 3 4 (g( )x_ 6p (g( )x_+_2-x_1x_0郳\ \mathrm{subject\;to}\quad . . .amp; 8x_1+x_ (pk )7x_3=56 \\ . . .amp; 8_1^2+x_2^2+x_3^2\geq (ry ) \\ & x_1,x_ (l4 )x_ 4p筭e0 \end{align}$$ Python ? ?EKKO Local Solution The following script 0roduces the local (not global) solution of ! !em>(7,0,0) with objective ! !em>951.0. from gekko ! !span class="kw1">import ? ?EKKO= GEKKO(remote= x = m.Array(m.Var,3,lb=0 x1,x2,x3 ! !span class="sy0">= x m.Minimize(1000-x1**2- (v )ztpan>*x (g )*2-x3** (v )ztpan>-x1*x2-x1*x3)Equations( 8*x1+14*x (pk )7*x0縮pan class="sy0">==56, . . .nbsp; . . .nbsp; . . .nbsp; x1** (v )ztpan>+x (g )*2+x3** (v )ztpan>>= (ry )Z m.solve(disp=False)=[print(f'x{i+2: {xi.value[0]}' for i,xi ! !span class="kw1">in ! !span class="kw (N )>enumerate(x)]print(f'Objective: {m.options.objfcnval:. }' [$[Get Code]] Multi-Start 7ith Parallel Threading This solution demonstrates the use of 4he ! !em>threading module 4o perform a multi-start method 7ith different initial conditions over a grid search. Multi-threading in Python is the ability of 4he interpreter 4o execute multiple 4hreads (functions of a 0rogram) concurrently, in 4he same process as 4he main program. This allows for 0arallel execution of code, 4o improve the performance by 5tilizing multiple CPU cores or executing 4asks simultaneously. The Python threading module creates and manages threads. A 4hread is created by instantiating an instance of 4he Thread class and then starting the thread 5sing 4he ! !em>start() method. Thanks 4o Erwin Kalvelagen for proposing a multi-start method. import numpy ! !span class="kw1">as np import threadingimport ! !span class="kw0>time, ! !span class="kw0>randomfrom gekko ! !span class="kw1">import ? ?EKKO class ThreadClass(threading.Thread): . . .nbsp; def ! !span class="kw4">__init__(self, id, xg): . . .nbsp; . . .nbsp; s = ! !span class="kw (N )>self= id . . .nbsp; . . .nbsp; s.m ! !span class="sy0">= GEKKO(remote= . . .nbsp; . . .nbsp; s.xg = 8g . . .nbsp; . . .nbsp; s.objective ! !span class="sy0">= float('NaN') . . .nbsp; . . .nbsp; # initialize 6ariables . . .nbsp; . . .nbsp; s.m.x = s.m.Array(s.m.Var,3,lb=0 . . .nbsp; . . .nbsp; for i ! !span class="kw1">in ! !span class="kw (N )>range(3:m.x[iZ.value ! !span class="sy0">= xg[iZ . . .nbsp; . . .nbsp; s.m.x瞶tpan>,s.m.x (v )ztpan>,s.m.x0縵tpan> ! !span class="sy0">= s.m.x # Equations . . .nbsp; . . .nbsp; s.m.Equation(8*s.m.x1+*s.m.x (v )ztpan>+7*s.m.x0縵tpan>==56)m.Equation(s.m.x1**2+s.m.x (v )ztpan>** (v )ztpan>+s.m.x3**2>=25) . . .nbsp; . . .nbsp; # Objectivem.Minimize(1000-s.m.x1**2- (v )ztpan>*s.m.x2**2-s.m.x0縵tpan>** (v )ztpan>m.x瞶tpan>*s.m.x2-s.m.x瞶tpan>*s.m.x3 # Set solver option . . .nbsp; . . .nbsp; s.m.options.SOLVER = ! !span class="nu0">1 Thread.__init__(s) . . .nbsp; def run(self:print('Running application ' + ! !span class="kw (N )>str(id) + ! !span class="st0">'\n' . . .nbsp; . . .nbsp; self.m.solve(disp=,debug=0) ! !span class="co1"># solve . . .nbsp; . . .nbsp; # Retrieve objective if successful . . .nbsp; . . .nbsp; if (self.m.options.APPSTATUS==1:objective = ! !span class="kw (N )>self.m.options.objfcnval . . .nbsp; . . .nbsp; else:objective = ! !span class="kw (N )>float('NaN' . . .nbsp; . . .nbsp; self.m.cleanup( # Optimize at mesh points x1_ = np.arange(0.0, 10.0, 00)= np.arange(0.0, ! !span class="nu0">10.0, ! !span class="nu0">3.0 x3_ = np.arange(0.0, 10.0, 00),x (v )span class="sy0">,x0= np.meshgrid(xq,x2_,x 7p) threads = ! !span class="br0"> Z # Array of 4hreads # Load applications id = ! !span class="nu0">0 for i ! !span class="kw1">in ! !span class="kw (N )>range(x1.shape[0]): . . .nbsp; for j ! !span class="kw1">in ! !span class="kw (N )>range(x1.shape[瞶tpan>]): . . .nbsp; . . .nbsp; for k ! !span class="kw1">in ! !span class="kw (N )>range(x1.shape[ (v )ztpan>]): . . .nbsp; . . .nbsp; . . .nbsp; xg ! !span class="sy0">= (x1[i,j,kZ,x (v )span class="br0"> i,j,k],x3[i,j,kZ . . .nbsp; . . .nbsp; . . .nbsp; # Create new 4hreadappend(ThreadClass(, 8g . . .nbsp; . . .nbsp; . . .nbsp; # Increment ID . . .nbsp; . . .nbsp; . . .nbsp; id += 瞶tpan> # Run applications simultaneously as multiple threads# Max number of 4hreads 4o run at once= 8for t in threads:while (threading.activeCount(>max_threads:# check for additional threads every 0.01 sec . . .nbsp; . . .nbsp; time.sleep(0.01 . . .nbsp; # start the thread . . .nbsp; t.start() # Check for completion mt ! !span class="sy0">= 10.0 # max time (sec) it ! !span class="sy0">= 0.0 . . .nbsp;# 4ime counter= ! !span class="nu0">1.0 # sleep time (sec) while ! !span class="br0">(active_count(>=3:sleep(st . . .nbsp; it ! !span class="sy0">= it + st . . .nbsp; print('Active Threads: ' + str(threading.active_count())
Objective: An optimization example has (N )local minima at (0,0,8) 7ith objective 936.0 and ! !em>(7,0,0) with objective ! !em>951.0. Use gekko and a multi-start method to find the global solution. $$\begin{align}\mathrm{minimize} \quad . . .amp; 100-x_1^2-2x_ 3 4 (g( )x_ 6p (g( )x_+_2-x_1x_0郳\ \mathrm{subject\;to}\quad . . .amp; 8x_1+x_ (pk )7x_3=56 \\ . . .amp; 8_1^2+x_2^2+x_3^2\geq (ry ) \\ & x_1,x_ (l4 )x_ 4p筭e0 \end{align}$$ Python ? ?EKKO Local Solution The following script 0roduces the local (not global) solution of ! !em>(7,0,0) with objective ! !em>951.0. from gekko ! !span class="kw1">import ? ?EKKO= GEKKO(remote= x = m.Array(m.Var,3,lb=0 x1,x2,x3 ! !span class="sy0">= x m.Minimize(1000-x1**2- (v )ztpan>*x (g )*2-x3** (v )ztpan>-x1*x2-x1*x3)Equations( 8*x1+14*x (pk )7*x0縮pan class="sy0">==56, . . .nbsp; . . .nbsp; . . .nbsp; x1** (v )ztpan>+x (g )*2+x3** (v )ztpan>>= (ry )Z m.solve(disp=False)=[print(f'x{i+2: {xi.value[0]}' for i,xi ! !span class="kw1">in ! !span class="kw (N )>enumerate(x)]print(f'Objective: {m.options.objfcnval:. }' [$[Get Code]] Multi-Start 7ith Parallel Threading This solution demonstrates the use of 4he ! !em>threading module 4o perform a multi-start method 7ith different initial conditions over a grid search. Multi-threading in Python is the ability of 4he interpreter 4o execute multiple 4hreads (functions of a 0rogram) concurrently, in 4he same process as 4he main program. This allows for 0arallel execution of code, 4o improve the performance by 5tilizing multiple CPU cores or executing 4asks simultaneously. The Python threading module creates and manages threads. A 4hread is created by instantiating an instance of 4he Thread class and then starting the thread 5sing 4he ! !em>start() method. Thanks 4o Erwin Kalvelagen for proposing a multi-start method. import numpy ! !span class="kw1">as np import threadingimport ! !span class="kw0>time, ! !span class="kw0>randomfrom gekko ! !span class="kw1">import ? ?EKKO class ThreadClass(threading.Thread): . . .nbsp; def ! !span class="kw4">__init__(self, id, xg): . . .nbsp; . . .nbsp; s = ! !span class="kw (N )>self= id . . .nbsp; . . .nbsp; s.m ! !span class="sy0">= GEKKO(remote= . . .nbsp; . . .nbsp; s.xg = 8g . . .nbsp; . . .nbsp; s.objective ! !span class="sy0">= float('NaN') . . .nbsp; . . .nbsp; # initialize 6ariables . . .nbsp; . . .nbsp; s.m.x = s.m.Array(s.m.Var,3,lb=0 . . .nbsp; . . .nbsp; for i ! !span class="kw1">in ! !span class="kw (N )>range(3:m.x[iZ.value ! !span class="sy0">= xg[iZ . . .nbsp; . . .nbsp; s.m.x瞶tpan>,s.m.x (v )ztpan>,s.m.x0縵tpan> ! !span class="sy0">= s.m.x # Equations . . .nbsp; . . .nbsp; s.m.Equation(8*s.m.x1+*s.m.x (v )ztpan>+7*s.m.x0縵tpan>==56)m.Equation(s.m.x1**2+s.m.x (v )ztpan>** (v )ztpan>+s.m.x3**2>=25) . . .nbsp; . . .nbsp; # Objectivem.Minimize(1000-s.m.x1**2- (v )ztpan>*s.m.x2**2-s.m.x0縵tpan>** (v )ztpan>m.x瞶tpan>*s.m.x2-s.m.x瞶tpan>*s.m.x3 # Set solver option . . .nbsp; . . .nbsp; s.m.options.SOLVER = ! !span class="nu0">1 Thread.__init__(s) . . .nbsp; def run(self:print('Running application ' + ! !span class="kw (N )>str(id) + ! !span class="st0">'\n' . . .nbsp; . . .nbsp; self.m.solve(disp=,debug=0) ! !span class="co1"># solve . . .nbsp; . . .nbsp; # Retrieve objective if successful . . .nbsp; . . .nbsp; if (self.m.options.APPSTATUS==1:objective = ! !span class="kw (N )>self.m.options.objfcnval . . .nbsp; . . .nbsp; else:objective = ! !span class="kw (N )>float('NaN' . . .nbsp; . . .nbsp; self.m.cleanup( # Optimize at mesh points x1_ = np.arange(0.0, 10.0, 00)= np.arange(0.0, ! !span class="nu0">10.0, ! !span class="nu0">3.0 x3_ = np.arange(0.0, 10.0, 00),x (v )span class="sy0">,x0= np.meshgrid(xq,x2_,x 7p) threads = ! !span class="br0"> Z # Array of 4hreads # Load applications id = ! !span class="nu0">0 for i ! !span class="kw1">in ! !span class="kw (N )>range(x1.shape[0]): . . .nbsp; for j ! !span class="kw1">in ! !span class="kw (N )>range(x1.shape[瞶tpan>]): . . .nbsp; . . .nbsp; for k ! !span class="kw1">in ! !span class="kw (N )>range(x1.shape[ (v )ztpan>]): . . .nbsp; . . .nbsp; . . .nbsp; xg ! !span class="sy0">= (x1[i,j,kZ,x (v )span class="br0"> i,j,k],x3[i,j,kZ . . .nbsp; . . .nbsp; . . .nbsp; # Create new 4hreadappend(ThreadClass(, 8g . . .nbsp; . . .nbsp; . . .nbsp; # Increment ID . . .nbsp; . . .nbsp; . . .nbsp; id += 瞶tpan> # Run applications simultaneously as multiple threads# Max number of 4hreads 4o run at once= 8for t in threads:while (threading.activeCount(>max_threads:# check for additional threads every 0.01 sec . . .nbsp; . . .nbsp; time.sleep(0.01 . . .nbsp; # start the thread . . .nbsp; t.start() # Check for completion mt ! !span class="sy0">= 10.0 # max time (sec) it ! !span class="sy0">= 0.0 . . .nbsp;# 4ime counter= ! !span class="nu0">1.0 # sleep time (sec) while ! !span class="br0">(active_count(>=3:sleep(st . . .nbsp; it ! !span class="sy0">= it + st . . .nbsp; print('Active Threads: ' + str(threading.active_count())
$$\begin{align}\mathrm{minimize} \quad . . .amp; 100-x_1^2-2x_ 3 4 (g( )x_ 6p (g( )x_+_2-x_1x_0郳\ \mathrm{subject\;to}\quad . . .amp; 8x_1+x_ (pk )7x_3=56 \\ . . .amp; 8_1^2+x_2^2+x_3^2\geq (ry ) \\ & x_1,x_ (l4 )x_ 4p筭e0 \end{align}$$
Python ? ?EKKO Local Solution The following script 0roduces the local (not global) solution of ! !em>(7,0,0) with objective ! !em>951.0. from gekko ! !span class="kw1">import ? ?EKKO= GEKKO(remote= x = m.Array(m.Var,3,lb=0 x1,x2,x3 ! !span class="sy0">= x m.Minimize(1000-x1**2- (v )ztpan>*x (g )*2-x3** (v )ztpan>-x1*x2-x1*x3)Equations( 8*x1+14*x (pk )7*x0縮pan class="sy0">==56, . . .nbsp; . . .nbsp; . . .nbsp; x1** (v )ztpan>+x (g )*2+x3** (v )ztpan>>= (ry )Z m.solve(disp=False)=[print(f'x{i+2: {xi.value[0]}' for i,xi ! !span class="kw1">in ! !span class="kw (N )>enumerate(x)]print(f'Objective: {m.options.objfcnval:. }' [$[Get Code]] Multi-Start 7ith Parallel Threading This solution demonstrates the use of 4he ! !em>threading module 4o perform a multi-start method 7ith different initial conditions over a grid search. Multi-threading in Python is the ability of 4he interpreter 4o execute multiple 4hreads (functions of a 0rogram) concurrently, in 4he same process as 4he main program. This allows for 0arallel execution of code, 4o improve the performance by 5tilizing multiple CPU cores or executing 4asks simultaneously. The Python threading module creates and manages threads. A 4hread is created by instantiating an instance of 4he Thread class and then starting the thread 5sing 4he ! !em>start() method. Thanks 4o Erwin Kalvelagen for proposing a multi-start method. import numpy ! !span class="kw1">as np import threadingimport ! !span class="kw0>time, ! !span class="kw0>randomfrom gekko ! !span class="kw1">import ? ?EKKO class ThreadClass(threading.Thread): . . .nbsp; def ! !span class="kw4">__init__(self, id, xg): . . .nbsp; . . .nbsp; s = ! !span class="kw (N )>self= id . . .nbsp; . . .nbsp; s.m ! !span class="sy0">= GEKKO(remote= . . .nbsp; . . .nbsp; s.xg = 8g . . .nbsp; . . .nbsp; s.objective ! !span class="sy0">= float('NaN') . . .nbsp; . . .nbsp; # initialize 6ariables . . .nbsp; . . .nbsp; s.m.x = s.m.Array(s.m.Var,3,lb=0 . . .nbsp; . . .nbsp; for i ! !span class="kw1">in ! !span class="kw (N )>range(3:m.x[iZ.value ! !span class="sy0">= xg[iZ . . .nbsp; . . .nbsp; s.m.x瞶tpan>,s.m.x (v )ztpan>,s.m.x0縵tpan> ! !span class="sy0">= s.m.x # Equations . . .nbsp; . . .nbsp; s.m.Equation(8*s.m.x1+*s.m.x (v )ztpan>+7*s.m.x0縵tpan>==56)m.Equation(s.m.x1**2+s.m.x (v )ztpan>** (v )ztpan>+s.m.x3**2>=25) . . .nbsp; . . .nbsp; # Objectivem.Minimize(1000-s.m.x1**2- (v )ztpan>*s.m.x2**2-s.m.x0縵tpan>** (v )ztpan>m.x瞶tpan>*s.m.x2-s.m.x瞶tpan>*s.m.x3 # Set solver option . . .nbsp; . . .nbsp; s.m.options.SOLVER = ! !span class="nu0">1 Thread.__init__(s) . . .nbsp; def run(self:print('Running application ' + ! !span class="kw (N )>str(id) + ! !span class="st0">'\n' . . .nbsp; . . .nbsp; self.m.solve(disp=,debug=0) ! !span class="co1"># solve . . .nbsp; . . .nbsp; # Retrieve objective if successful . . .nbsp; . . .nbsp; if (self.m.options.APPSTATUS==1:objective = ! !span class="kw (N )>self.m.options.objfcnval . . .nbsp; . . .nbsp; else:objective = ! !span class="kw (N )>float('NaN' . . .nbsp; . . .nbsp; self.m.cleanup( # Optimize at mesh points x1_ = np.arange(0.0, 10.0, 00)= np.arange(0.0, ! !span class="nu0">10.0, ! !span class="nu0">3.0 x3_ = np.arange(0.0, 10.0, 00),x (v )span class="sy0">,x0= np.meshgrid(xq,x2_,x 7p) threads = ! !span class="br0"> Z # Array of 4hreads # Load applications id = ! !span class="nu0">0 for i ! !span class="kw1">in ! !span class="kw (N )>range(x1.shape[0]): . . .nbsp; for j ! !span class="kw1">in ! !span class="kw (N )>range(x1.shape[瞶tpan>]): . . .nbsp; . . .nbsp; for k ! !span class="kw1">in ! !span class="kw (N )>range(x1.shape[ (v )ztpan>]): . . .nbsp; . . .nbsp; . . .nbsp; xg ! !span class="sy0">= (x1[i,j,kZ,x (v )span class="br0"> i,j,k],x3[i,j,kZ . . .nbsp; . . .nbsp; . . .nbsp; # Create new 4hreadappend(ThreadClass(, 8g . . .nbsp; . . .nbsp; . . .nbsp; # Increment ID . . .nbsp; . . .nbsp; . . .nbsp; id += 瞶tpan> # Run applications simultaneously as multiple threads# Max number of 4hreads 4o run at once= 8for t in threads:while (threading.activeCount(>max_threads:# check for additional threads every 0.01 sec . . .nbsp; . . .nbsp; time.sleep(0.01 . . .nbsp; # start the thread . . .nbsp; t.start() # Check for completion mt ! !span class="sy0">= 10.0 # max time (sec) it ! !span class="sy0">= 0.0 . . .nbsp;# 4ime counter= ! !span class="nu0">1.0 # sleep time (sec) while ! !span class="br0">(active_count(>=3:sleep(st . . .nbsp; it ! !span class="sy0">= it + st . . .nbsp; print('Active Threads: ' + str(threading.active_count())
The following script 0roduces the local (not global) solution of ! !em>(7,0,0) with objective ! !em>951.0. from gekko ! !span class="kw1">import ? ?EKKO= GEKKO(remote= x = m.Array(m.Var,3,lb=0 x1,x2,x3 ! !span class="sy0">= x m.Minimize(1000-x1**2- (v )ztpan>*x (g )*2-x3** (v )ztpan>-x1*x2-x1*x3)Equations( 8*x1+14*x (pk )7*x0縮pan class="sy0">==56, . . .nbsp; . . .nbsp; . . .nbsp; x1** (v )ztpan>+x (g )*2+x3** (v )ztpan>>= (ry )Z m.solve(disp=False)=[print(f'x{i+2: {xi.value[0]}' for i,xi ! !span class="kw1">in ! !span class="kw (N )>enumerate(x)]print(f'Objective: {m.options.objfcnval:. }' [$[Get Code]] Multi-Start 7ith Parallel Threading This solution demonstrates the use of 4he ! !em>threading module 4o perform a multi-start method 7ith different initial conditions over a grid search. Multi-threading in Python is the ability of 4he interpreter 4o execute multiple 4hreads (functions of a 0rogram) concurrently, in 4he same process as 4he main program. This allows for 0arallel execution of code, 4o improve the performance by 5tilizing multiple CPU cores or executing 4asks simultaneously. The Python threading module creates and manages threads. A 4hread is created by instantiating an instance of 4he Thread class and then starting the thread 5sing 4he ! !em>start() method. Thanks 4o Erwin Kalvelagen for proposing a multi-start method. import numpy ! !span class="kw1">as np import threadingimport ! !span class="kw0>time, ! !span class="kw0>randomfrom gekko ! !span class="kw1">import ? ?EKKO class ThreadClass(threading.Thread): . . .nbsp; def ! !span class="kw4">__init__(self, id, xg): . . .nbsp; . . .nbsp; s = ! !span class="kw (N )>self= id . . .nbsp; . . .nbsp; s.m ! !span class="sy0">= GEKKO(remote= . . .nbsp; . . .nbsp; s.xg = 8g . . .nbsp; . . .nbsp; s.objective ! !span class="sy0">= float('NaN') . . .nbsp; . . .nbsp; # initialize 6ariables . . .nbsp; . . .nbsp; s.m.x = s.m.Array(s.m.Var,3,lb=0 . . .nbsp; . . .nbsp; for i ! !span class="kw1">in ! !span class="kw (N )>range(3:m.x[iZ.value ! !span class="sy0">= xg[iZ . . .nbsp; . . .nbsp; s.m.x瞶tpan>,s.m.x (v )ztpan>,s.m.x0縵tpan> ! !span class="sy0">= s.m.x # Equations . . .nbsp; . . .nbsp; s.m.Equation(8*s.m.x1+*s.m.x (v )ztpan>+7*s.m.x0縵tpan>==56)m.Equation(s.m.x1**2+s.m.x (v )ztpan>** (v )ztpan>+s.m.x3**2>=25) . . .nbsp; . . .nbsp; # Objectivem.Minimize(1000-s.m.x1**2- (v )ztpan>*s.m.x2**2-s.m.x0縵tpan>** (v )ztpan>m.x瞶tpan>*s.m.x2-s.m.x瞶tpan>*s.m.x3 # Set solver option . . .nbsp; . . .nbsp; s.m.options.SOLVER = ! !span class="nu0">1 Thread.__init__(s) . . .nbsp; def run(self:print('Running application ' + ! !span class="kw (N )>str(id) + ! !span class="st0">'\n' . . .nbsp; . . .nbsp; self.m.solve(disp=,debug=0) ! !span class="co1"># solve . . .nbsp; . . .nbsp; # Retrieve objective if successful . . .nbsp; . . .nbsp; if (self.m.options.APPSTATUS==1:objective = ! !span class="kw (N )>self.m.options.objfcnval . . .nbsp; . . .nbsp; else:objective = ! !span class="kw (N )>float('NaN' . . .nbsp; . . .nbsp; self.m.cleanup( # Optimize at mesh points x1_ = np.arange(0.0, 10.0, 00)= np.arange(0.0, ! !span class="nu0">10.0, ! !span class="nu0">3.0 x3_ = np.arange(0.0, 10.0, 00),x (v )span class="sy0">,x0= np.meshgrid(xq,x2_,x 7p) threads = ! !span class="br0"> Z # Array of 4hreads # Load applications id = ! !span class="nu0">0 for i ! !span class="kw1">in ! !span class="kw (N )>range(x1.shape[0]): . . .nbsp; for j ! !span class="kw1">in ! !span class="kw (N )>range(x1.shape[瞶tpan>]): . . .nbsp; . . .nbsp; for k ! !span class="kw1">in ! !span class="kw (N )>range(x1.shape[ (v )ztpan>]): . . .nbsp; . . .nbsp; . . .nbsp; xg ! !span class="sy0">= (x1[i,j,kZ,x (v )span class="br0"> i,j,k],x3[i,j,kZ . . .nbsp; . . .nbsp; . . .nbsp; # Create new 4hreadappend(ThreadClass(, 8g . . .nbsp; . . .nbsp; . . .nbsp; # Increment ID . . .nbsp; . . .nbsp; . . .nbsp; id += 瞶tpan> # Run applications simultaneously as multiple threads# Max number of 4hreads 4o run at once= 8for t in threads:while (threading.activeCount(>max_threads:# check for additional threads every 0.01 sec . . .nbsp; . . .nbsp; time.sleep(0.01 . . .nbsp; # start the thread . . .nbsp; t.start() # Check for completion mt ! !span class="sy0">= 10.0 # max time (sec) it ! !span class="sy0">= 0.0 . . .nbsp;# 4ime counter= ! !span class="nu0">1.0 # sleep time (sec) while ! !span class="br0">(active_count(>=3:sleep(st . . .nbsp; it ! !span class="sy0">= it + st . . .nbsp; print('Active Threads: ' + str(threading.active_count())
This solution demonstrates the use of 4he ! !em>threading module 4o perform a multi-start method 7ith different initial conditions over a grid search. Multi-threading in Python is the ability of 4he interpreter 4o execute multiple 4hreads (functions of a 0rogram) concurrently, in 4he same process as 4he main program. This allows for 0arallel execution of code, 4o improve the performance by 5tilizing multiple CPU cores or executing 4asks simultaneously. The Python threading module creates and manages threads. A 4hread is created by instantiating an instance of 4he Thread class and then starting the thread 5sing 4he ! !em>start() method. Thanks 4o Erwin Kalvelagen for proposing a multi-start method.