Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
optimization-parameter-scaling [2017/11/13 09:54] awfoptimization-parameter-scaling [2021/09/03 13:34] (current) awf
Line 36: Line 36:
 So instead I far prefer to place some onus on you, the user of the routine.  Your job is this: So instead I far prefer to place some onus on you, the user of the routine.  Your job is this:
  
-> *Scale x so its values are "around 1"*+**Scale x so its components are "around 1"**
  
 This is actually pretty easy in practice.   This is actually pretty easy in practice.  
Line 49: Line 49:
 the objective also changes by a reasonable amount.    the objective also changes by a reasonable amount.   
 So if one of the parameters is phase of the moon, it might feel natural to  So if one of the parameters is phase of the moon, it might feel natural to 
-measure it in days (029.5) with some handling for wraparound.  +measure it in days (0 < phoon < 29.5) with some handling for wraparound.  
 Then your labmate points out that the SI unit for time is the second so you dutifully reparameterize it in seconds, in the range (0, 2500000) Then your labmate points out that the SI unit for time is the second so you dutifully reparameterize it in seconds, in the range (0, 2500000)
 But if you add 1e-5 seconds, it will probably change the objective by very little, maybe by 1e-12 gigawatts, and the optimizer will be in trouble. But if you add 1e-5 seconds, it will probably change the objective by very little, maybe by 1e-12 gigawatts, and the optimizer will be in trouble.
  
-Your days were a better choice (about 86400 times better), giving a change of 1e-7 GW when you add 1e-5 days (about a second).+Your first idea (do it in days) was actually a better choice (about 86400 times better), giving a change of 1e-7 GW when you add 1e-5 days (about a second).  Can we do even better?
  
-Ultimately you're probably anyway looking at terms in the objective which include the phase of the moon as+Well, look at how the parameter appears in the objective funciton.  For phase of the moon, that's normally in some term like:
  
-  cos(eta * phoon)^beta+$$  \cos(\eta * \mathtt{phoon})^\beta  $$
  
-so perhaps a range of 0..1/eta is sensible.   +so a sensible range might be to $2\pi/\eta$.   
-The point is just to think about how a change in phoon affects the objective.+The point is just to think about how a change in ''phoon'' affects the objective.
  
-=== But that's the easy case, what about dimensionless parameters? ===+But that's the easy case, what about dimensionless parameters?
  
 Yeah, not an issue.   Imagine that beta above is also part of the optimization. ((It's quite likely we would need beta,  Yeah, not an issue.   Imagine that beta above is also part of the optimization. ((It's quite likely we would need beta, 
Line 77: Line 77:
 $\def\e#1#2{{\mathbf e}^{#1}_{#2}}$ $\def\e#1#2{{\mathbf e}^{#1}_{#2}}$
  
-=== Another reason for good scaling: finite difference Jacobian calculation ===+Another reason for good scaling: finite difference Jacobian calculation
  
 In many situations, you aren't yet sure enough of your model function $\F(x)$ to have  In many situations, you aren't yet sure enough of your model function $\F(x)$ to have 
Line 85: Line 85:
  
 Let's look at one entry: $J_{ij} = \frac{\partial F_i}{\partial x_j}$, and approximate it using the limit definition.   Let's look at one entry: $J_{ij} = \frac{\partial F_i}{\partial x_j}$, and approximate it using the limit definition.  
-The vector $\e nk$ is the unit vector in $\R^n$ with a one in the $k^\text{th}$ element and zeros elsewhere, +The vector $\e nj$ is the unit vector in $\R^n$ with a one in the $j^\text{th}$ element and zeros elsewhere, 
 and $F_i(x)$ is the $i^\text{th}$ component of $\F(x)$. and $F_i(x)$ is the $i^\text{th}$ component of $\F(x)$.
  
Line 108: Line 108:
  
 So.  In conclusion.  Think about the parameters, and try to keep them in sensible units. So.  In conclusion.  Think about the parameters, and try to keep them in sensible units.
 +
optimization-parameter-scaling.1510566852.txt.gz · Last modified: 2017/11/13 09:54 by awf
CC Attribution 4.0 International
Driven by DokuWiki Recent changes RSS feed Valid CSS Valid XHTML 1.0