Class Hierarchy for Generic Parameter Optimization Problems

Question

I am working on a code library containing data structures and algorithms for solving parameter optimization problems. A parameter optimization problem is a problem of the form: given a vector of parameters P, and a loss function L(P), find the vector components of P such that L(P) is minimized. The implementation of the parameters and the loss function are specific to the problem, but they can all be represented as floating point data types. So far I have an abstract base class called Parameters like so:

public abstract class Parameters
{
 /// <summary>
 /// Maps the names of the parameters to the values.
 /// </summary>
 public IImmutableDictionary<string, double> Params { get; }
 public Parameters(IImmutableDictionary<string, double> parameters)
 {
 Params = parameters;
 }
 /// <summary>
 /// Returns a string representation of the parameters.
 /// </summary>
 public override string ToString()
 {
 string parametersStr = $"{String.Join('\n', Params.Select(p => $"{p.Key} = {p.Value}"))}";
 return parametersStr;
 }
}

I also have a definition of a loss function like so:

/// <summary>
/// Calculates a fitness score for the given parameters.
/// <returns>The fitness of the parameters (higher values are better).</returns>
public delegate double FitnessFunction(Parameters parameters);

I can then define the contract for an IParameterOptimizer like so:

/// <summary>
/// Represents an object capable of optimizing parameters.
/// </summary>
public interface IParameterOptimizer
{
 /// <summary>
 /// Optimizes parameters using the given fitness function.
 /// </summary>
 /// <returns>A list of 2-tuples of parameters and fitness scores.</returns>
 public Task<IList<(Parameters, double)>> OptimizeParameters(FitnessFunction fitnessFunction);
}

So an object implementing IParameterOptimizer can return an ordered list of tuples mapping parameters to loss values, with more fit values appearing later in the list.

Now consider as a concrete example, a SalesParameters class, and a MaximizeProfit fitness function like so:

public class SalesParameters : Parameters
{
 public int ManufacturedUnits { get; }
 public double Price { get; }
 public double MarketingCost { get; }
 public SalesParameters(int units, double price, double marketing) : 
 base(new Dictionary<string, double>() {
 { "ManufacturedUnits", (double)ManufacturedUnits },
 { "Price", Price },
 { "MarketingCost", MarketingCost },
 }.ToImmutableDictionary())
 {
 ManufacturedUnits = units;
 Price = price;
 MarketingCost = marketing;
 }
}
public double MaximizeProfit(SalesParameters p)
{
 return (p.ManufacturedUnits * p.Price) - p.MarketingCost;
}

The consumer of my library could then call one of my many implementations of IParameterOptimizer to find SalesParameters that will maximize their profits.

My question is - is this a sensible design? I find the Parameters dictionary mapping parameter names to values to be necessary to allow the optimization algorithms to know the type of operations that can be performed on the parameters (addition, subtraction, etc.). But it seems a little confusing and cumbersome for the user of the library to need to subclass Parameters and set the Params dictionary. Still, I can't think of a better solution as I would like to avoid using reflection.

• Doc Brown: "I find the Parameters dictionary mapping parameter names to values to be necessary to allow the optimization algorithms to know the type of operations that can be performed on the parameters" - how exactly? By their name? Honestly, it looks like you did not tell us the full story.
• Doc Brown: ... 2) Does your optimizer really generate an IList<(Parameters, double)? In optimization, the goal is usually to find one (optimal or suboptimal) Parameters instance - what is your idea of generating many? Different local optima? A sequence of intermediate optima? Please clarify.
• JonasH: How is the optimizer supposed to generate new parameter objects? The posted example looks a bit half baked, I don't really see how it could be used in practice. I would take a look at how existing math libraries handle optimization. I would also consider things like parameter constraints and/or starting position.
• Doc Brown: @Treker: the interesting question is here, if the mutation shall be part of the generic optimizer, or if it is part of the specific Parameter object (so a SalesParameters object might create a new, mutated SalesParameters object). Will the mutated object share the same IImmutableDictionary with the original object? I think this would make sense, so the description will only exist once in mem, which was a point of the design I would otherwise have questioned.
• Doc Brown: ... Another thing which isn't clear to me is why you need this IImmutableDictionary. For the current case, it looks overcomplicated - a simple, specific ToString() implementation in each derivation of Parameters would IMHO be simpler and sufficient. But maybe you left something out where the IImmutableDictionary becomes more useful?

Answer 1

I think the simplest and clearest solution is to avoid subclassing altogether. Instead, my package will contain a concrete Parameters class like so ->

public class Parameters
{
 /// <summary>
 /// Contains real number parameter values.
 /// </summary>
 public IImmutableList<double> Params { get; }
 public Parameters(IImmutableList<double> parameters)
 {
 Params = parameters;
 }
}

The consumer of my package will then be responsible for converting their problem specific data (SalesParameters in my example) into a Parameters object to interface with an IParameterOptimizer implementation.

This solution has the advantage of decoupling the data structures of the specific problem domain from the data structures and algorithms of the parameter optimization package. The only disadvantage is the consumer must write code to convert their specific object properties into a IImmutableList to instantiate a Parmaters object.

• Doc Brown: It makes things definitely easier to handle if you can start with something simple if it is sufficient, and separate the general optimization algoriíthm from the specific problem domain. Still, it is quite impossible to say if this design is useful or not with knowing how the generic Optimizer will use those parameter objects, how much it depends on the parameters being a vector (list) of double, or if that is just an unimportant implementation detail.
• Treker: Different IParameterOptimizer implementations use the parameter vector differently. For example: - BruteForceParameterOptimizer -> receives an IEnumerable<Parameters> of parameter combinations to test. - GeneticAlgorithmParameterOptimizer -> receives an initial generation of Parameters, as well as a mutation function - public delegate Parameters mutate(Parameters parentGeneration); - GeneticBreedingParameterOptimizer -> receives two initial generations, as well as a breeding function - public delegate IEnumerable<Parameters> Breed(Parameters parent1, Parameters parent2)
• Doc Brown: Is there any of these optimizers which require the parameter values to be stored in a specific data structure (like IImmutableList<double> in this answer)?
• Treker: No, so I changed it to IReadOnlyList<double> to be more generic.
• Doc Brown: My point is: why is "Parameters" a class? Why not just an interface? If your mutators are specific, the fitness function is specific, maybe with a specific ToString, what is the point of holding the values directly inside Parameters? Why not make it an interface IParameters?