Showing posts with label Microsoft Solver Foundation. Show all posts
Showing posts with label Microsoft Solver Foundation. Show all posts

Friday, October 3, 2014

Calibration of short rate models in Excel with C#, Solver Foundation and Excel-DNA

This time, I wanted to present one possible solution for calibrating one-factor short interest rate model to market data. As we already know, generalized form for stochastic differential equation (SDE) for any one-factor short interest rate model is the following.




Where u and w are still undefined functions of r and t. This SDE must be solved either analytically (if such solution exists) or numerically (Monte Carlo, FDM). After solving the model, it can then be used to price different interest rate products.

The problem in these models lies in the model parameters (u, w). We could estimate those parameters from the set of market data, but in this case we (most probably) end up with a set of resulting theoretical bond prices, not matching with the corresponding market bond prices. In the case of valuing or hedging interest rate products, such prices would be then more or less useless. The second approach is to turn the whole scheme upside down. Instead of estimating parameters from the set of market data directly and then feeding those into model, we would solve a set of parameters in such a way, that our resulting theoretical bond prices will match exactly with the corresponding market bond prices. In this posting, we will solve that set of parameters numerically.

Just for a reference, full derivation of Bond Pricing Equation (BPE), plus some solutions for tractable short rate models are presented in the book written by Paul Wilmott, for example.

PROJECT OUTCOME

The end product of this small project will be C# program, which will solve numerically time-dependent theta parameters for a given short rate model (Ho-Lee as our example case) for a given set of market zero-coupon bond prices, assuming that time-dependent theta parameters are piecewise constant. C# program is using Microsoft Solver Foundation for performing the required optimization tasks. Finally, we will interface C# program into Excel workbook via VBA, with Excel-DNA.

ANALYTICAL SOLUTION FOR HO-LEE SHORT RATE MODEL


Ho-Lee model was the first no-arbitrage-type short rate model, which could be calibrated to market data. In this first section, we will go through the process of solving this SDE analytically. SDE for Ho-Lee model is the following.





Where theta is time-dependent drift coefficient (u), sigma is constant diffusion coefficient (w) and dx is the standard brownian motion. By replacing these coefficients into bond pricing equation (BPE), we get the following partial differential equation (PDE).






For this PDE, we are (very fortunately) looking for very specific solution, having the following functional form.




To solve our PDE above, we will substitute the previous solution into PDE. First, we calculate the following partial derivatives. Theta (in this case, partial derivative to time) is the following.






Delta is the following.






Gamma is the following.


After calculating partial derivatives, we substitute these into original BPE and divide all terms with V. Now we have the following PDE to be solved.






After separating all r-dependent terms and r-independent terms, we will have two ordinary differential equations (ODE) to be solved. For B coefficient, ODE is the following.





For A coefficient, ODE is the following.






After integrating ODE for B coefficient from t to T, we have the following solution.




After integrating ODE for A coefficient from t to T and substituting B coefficient into equation, we have the following solution.






We could still continue and find the exact analytical solution for time-dependent theta coefficient from this integral equation. However, for any real-life calibration purposes we have to make some assumptions concerning theta parameter. For the sake of easiness in calculations, we assume this time-dependent theta parameter to be piecewise constant. With this assumption, we get the following solution for A coefficient.






After integrating the last existing integral term, we have the following solution for A coefficient.





Finally, we have the following functional form for a zero-coupon bond price according to Ho-Lee, which can be used for calibration purposes.






From this equation, we could also solve for the unknown theta parameter analytically, without any numerical methods involved.







CALIBRATION

The pricing process with models usually starts with some already known parameters, which are then feeded into some model to calculate the results. For example, in the case of Ho-Lee model, we would know the values for spot rate, time-dependent theta, constant sigma and bond maturity. Then, by calculating coefficients A and B we would calculate zero-coupon bond price and the corresponding yield.

Concerning short rate models, these parameters are usually not known in advance and have to be estimated or calibrated. In the calibration process, we are performing the whole pricing process backwards. First, we know the market yield for a zero-coupon bond. From this yield we calculate the corresponding bond price. Assuming that we know (or have been estimating) the values for spot rate, maturity and constant sigma parameter, we can then solve for coefficient A and B. Finally, we can calculate the value for time-dependent theta parameter.

Consider the case, in which we have Libor zero curve as our input market data, consisting of N zero-coupon bond yields. From this curve, we calculate the corresponding N zero-coupon bond prices. Consider also, that we have statistically estimated the value for constant sigma (standard deviation of a short rate) and we know spot rate and N maturities. After this, we set N initial values for our piecewise constant theta parameters. By using these theta parameters, we then calculate N pairs of A and B coefficients. Finally, N theoretical bond prices and N theoretical yields can be calculated by using N pairs of A and B coefficients. In the final stage, we just adjust our initial values for our N piecewise constant theta parameters, until theoretical yields and corresponding market yields are equal for all N bonds. More specifically, we minimize the sum of absolute squared errors between theoretical yields and corresponding market yields. This part is just unconstrained minimization problem. As a result of this minimization we will solve N theta parameters, for which theoretical bond prices are matching exactly with market bond prices. In other words, we are calibrating our model to market data.

As already noted above, we are dealing with the model having analytical solution (affine or tractable models). In this case, there is actually no need for any numerical method when calibrating model parameters to market. However, for the sake of curiosity and craftmanship, we will perform this calibration process numerically.


PREPARATORY TASKS

Next, we have to get the tools for performing optimization needed in the calibration process. As a proud holder of the NAG licence, I would use these numerical libraries for the task. However, in this project we will use Microsoft Solver Foundation, since it is freely available without any costs involved. Next, download Solver Foundation solver (32-bit). If this package is completely alien for you, there are excellent hands-on tutorials available, written by Mathias Brandewinder in here and in here. Replicating and studying these examples will get you started with Solver Foundation.

Also, download and unzip Excel-DNA Version 0.30 zip file to be ready when needed. This is the version I have been using so far, but I assume you can also use newer version 0.32. Concerning the tools needed, we are now ready to move on to create the actual C# program.


PROGRAM DESIGN

Design for C# program has been presented in the following UML chart. In order to understand better what is happening between the objects, some comments are given inside the blue boxes. The main point in this design is its flexibility to handle new one-factor models. This example has been specifically made using Ho-Lee model. However, user is able to create new implementations for CIR or Vasicek model, if needed.



















C# PROGRAM

Create a new C# Class project "SolverApp_25092014". Target framework is .NET Framework 4. Remember also to create a new reference into SolverFoundation dll file (Project - Add reference - Browse - \Microsoft.Solver.Foundation.dll). My dll file is found within the following folder: C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0\Microsoft.Solver.Foundation.dll. The workhorse classes for this program are presented below. Create new class files and copyPaste the following codes into separate classes.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Microsoft.SolverFoundation.Services;

namespace SolverApp_25092014
{
    public interface IOneFactorModel
    {
        Term price(Term parameter, Term t);
    }
}


using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Microsoft.SolverFoundation.Services;

namespace SolverApp_25092014
{
    public abstract class OneFactorModel : IOneFactorModel
    {
        public abstract Term price(Term parameter, Term t);
    }
}


using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Microsoft.SolverFoundation.Services;

namespace SolverApp_25092014
{
    public class HoLee : OneFactorModel
    {
        private Term spot;
        private Term stdev;
        //
        public HoLee(double spot, double stdev)
        {
            this.stdev = stdev; this.spot = spot;
        }
        public override Term price(Term parameter, Term t)
        {
            // create term object for Ho-Lee bond price for a given eta and t
            Term B = t;
            Term c1 = Model.Quotient(1, 2);
            Term c2 = Model.Quotient(1, 6);
            Term A = -(parameter * (c1 * Model.Power(t, 2))) + c2 * Model.Power(stdev, 2) * Model.Power(t, 3);
            return Model.Exp(A - spot * B);
        }
    }
}


using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Microsoft.SolverFoundation.Services;

namespace SolverApp_25092014
{
    public class OneFactorCalibrator
    {
        private OneFactorModel model;
        private Dictionary<double, double> curve;
        //
        public OneFactorCalibrator(OneFactorModel model, Dictionary<double, double> curve)
        {
            this.model = model; this.curve = curve;
        }
        public List<double> calibrate()
        {
            // create a new solver model
            SolverContext solver = new SolverContext();
            solver.ClearModel();
            Model solverModel = solver.CreateModel();
            //
            // create decision variables, a set of parameters to be calibrated
            var decisions = curve.Select(it => new Decision(Domain.Real, "parameter_" + it.Key.ToString()));
            solverModel.AddDecisions(decisions.ToArray());
            //
            // create objective function as sum of differences between market and theoretical yields
            SumTermBuilder terms = new SumTermBuilder(curve.Count);
            int i = 0;
            foreach (KeyValuePair<double, double> kvp in curve)
            {
                // define required term objects
                Term eta = solverModel.Decisions.ElementAt(i++);
                Term t = kvp.Key;
                //
                // calculate bond prices and yields
                Term theoreticalPrice = model.price(eta, t);
                Term theoreticalYield = Model.Power(Model.Quotient(1, theoreticalPrice), Model.Quotient(1, t)) - 1;
                Term marketPrice = kvp.Value;
                Term marketYield = Model.Power(Model.Quotient(1, marketPrice), Model.Quotient(1, t)) - 1;
                Term yieldDifference = Model.Power(marketYield - theoreticalYield, 2);
                //
                // add constructed term into sumbuilder (objective function)
                terms.Add(yieldDifference);
            }
            //
            // define optimization goal, solve the model and pack results into list
            solverModel.AddGoal("solverGoal", GoalKind.Minimize, Model.Abs(terms.ToTerm()));
            Solution result = solver.Solve();
            return solverModel.Decisions.Select(it => it.ToDouble()).ToList();
        }
    }
}

At this point, we should be able to build this class project without any errors or warnings. The only thing that might create a bit of confusion here, is the logic of using Solver Foundation and especially its Term object. If you feel uncomfortable using solver objects, you may spend some quality time and go through examples by Mathias Brandewinder mentioned earlier. Also, going through the stuff presented in Microsoft.Solverfoundation.Services Namespace relieves the painful climbing on this new learning curve. There are also some documentation pdf files included in the downloaded package from Microsoft. Anyway, the basic functionality for calibration process has now been set. Next, we have to interface our C# program and Excel.


Excel-DNA

Concerning the usage and creating interface between Excel and C#, we are following instructions presented in this blog posting.

Add reference to Excel-DNA library (Project - Add reference - Browse - \\ExcelDna.Integration.dll) and click OK. This dll file is inside the distribution folder what we just downloaded from Excel-DNA  website. From the properties of this reference, set Copy Local to be False.

Add new file as text file to project (Project - Add new item - Text file) and name it to be SolverApp_25092014.dna. CopyPaste the following xml code into this file.

<DnaLibrary Name="SolverApp_25092014" RuntimeVersion="v4.0">
  <ExternalLibrary Path="SolverApp_25092014.dll" />
</DnaLibrary>

From the properties of this dna file, set Copy to Output Directory to be Copy if newer.

Next, from the downloaded Excel-DNA folder (Distribution), copy ExcelDna.xll file into your project folder (\\Projects\SolverApp_25092014\SolverApp_25092014) and rename it to be SolverApp_25092014.xll. Then, add this xll file into your current project (Project - Add existing item). At this point, it might be that you do not see anything else, except cs files on this window. From drop down box on the bottom right corner of this window, select All files and you should see ExcelInterface.xll file what we just pasted into this ExcelInterface folder. Select this file and press Add. Finally, from the properties of this xll file, set Copy to Output Directory to be Copy if newer.

Build the solution. Again, everything should have gone well without any errors or warnings. At this point, my \\SolverApp_25092014\bin\Release folder looks like the following.





Next, we start to build interface between C# program and Excel workbook, via VBA


EXCEL USER INTERFACE AND C#-VBA INTEGRATION

Add new class into project and name it to be "SolverApp_25092014". Remember also, that we might need to use Message Box for error reporting coming from catch block. Hence, we need to create reference to System.Windows.Forms library (Project - Add reference - .NET - System.Windows.Forms). CopyPaste the following code into this new class file. After this, build solution again and save it. We should be able to rebuild this class project again without any errors or warnings.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using ExcelDna.Integration;

namespace SolverApp_25092014
{
    public static class SolverApp_25092014
    {
        public static void calibrate()
        {
            try
            {
                // create Excel application object
                dynamic Excel = ExcelDnaUtil.Application;
                //
                // read scalar parameters from named ranges
                double r = (double)Excel.Range["_r"].Value2;
                double stdev = (double)Excel.Range["_stdev"].Value2;
                //
                // read curve from named range
                dynamic Matrix = Excel.Range["_curve"].Value2;
                int rows = Matrix.GetUpperBound(0);
                //
                // export curve data into dictionary
                Dictionary<double, double> curve = new Dictionary<double, double>();
                for (int i = 0; i < rows; i++)
                {
                    double key = (double)Matrix.GetValue(i + 1, 1);
                    double value = (double)Matrix.GetValue(i + 1, 2);
                    curve.Add(key, value);
                }
                //
                // create model and calibrator instances
                OneFactorModel model = new HoLee(r, stdev);
                OneFactorCalibrator calibrator = new OneFactorCalibrator(model, curve);
                List<double> theta = calibrator.calibrate();
                //
                // export theta parameters into 2-dimensional array
                double[,] result = new double[rows, 1];
                for (int i = 0; i < theta.Count; i++) result[i, 0] = theta[i];
                //
                // print array into named range
                Excel.Range["_theta"] = result;
            }
            catch (Exception e)
            {
                MessageBox.Show(e.Message.ToString());
            }
        }
    }
}


We are almost there. Next, we will create Excel worksheet interface and VBA "trigger program" for C# program. The action starts as the user is pressing commandbutton, which then calls a simple VBA program, which then triggers the actual C# workhorse program. Create the following data view into a new Excel worksheet. Also, insert command button into worksheet.



















A few words concerning this worksheet. Zero-coupon bond curve maturities and prices are given in the columns B and C (yellow). Corresponding yields are calculated in the column D. Vector of time-dependent piecewise constant thetas will be calibrated and printed into column F (yellow). Columns from G to K are using analytical solution for Ho-Lee model to calculate theoretical zero-coupon bond prices and yields. Squared differences between market yields and calculated theoretical yields are given in the column K.

Input (output) data for (from) C# program are going to be read (written) directly from (to) Excel named ranges. Create the following named Excel ranges.






















Finally, insert a new standard VBA module and create simple program calling the actual C# program (Sub tester). Also, create event-handling program for commandbutton click, which then calls the procedure shown below.














Concerning the program and its interfacing with Excel, all those small pieces have finally been assembled together. Now, it is time to do the actual calibration test run.


TEST RUN AND MODEL VALIDATION

In this section we will perform test run and check, whether our calibration model is working properly. In another words, are we able to replicate the market with our model. After pressing "calibrate" button, my Excel shows the following calibration results for theta parameters shown in column F.





















Analytical results are already calculated in the worksheet (columns G to J). For example, time-dependent theta parameter for 4 years is 0.00771677. With this theta value embedded in A coefficient will produce zero-coupon bond price, equal to corresponding market price.

How about using SDE directly with these calibrated time-dependent thetas? Let us use Monte Carlo to price 4-year zero-coupon bond by using SDE directly. For the sake of simplicity, we perform this exercise in Excel. Simulation scheme is presented in the screenshot below.




















In this Excel worksheet presented above, 4-year period has been divided into 800 time steps, which gives us the size for one time step to be 0.005 years. Total of 16 378 paths was simulated by using Ho-Lee SDE with calibrated time-dependent piecewise constant theta parameter. All simulated zero-coupon bond prices have been calculated in the row 805. By taking the average for all these 16 378 simulated zero-coupon bond prices, gives us the value of 0.9343. Needless to say this value is  random variable, which changes every time we press that F9 button. Our corresponding market price for 4-year zero-coupon bond is 0.9344, so we are not far away from that value. We can make the conclusion, that our calibration is working as expected.

Thanks for reading this blog.
-Mike

Friday, June 21, 2013

Using MS Solver Foundation and C# in Excel with Excel-DNA

In one of my latest posts http://mikejuniperhill.blogspot.fi/2013/06/using-excel-solver-with-vba.html I was presenting my approach for using Frontline Solver in Excel with VBA. I was also mentioning, that I would like to get rid of linking my program with concrete ranges in Excel worksheet. Why? To be able to create convinient User Defined Functions (UDF) for solving optimization routines. I would like to have an optimization tool, in which I take all given parameters and data inside to a program and perform any desired optimization routines inside a program, without interacting with Excel ranges. Now it is a good time to present one such a tool - Microsoft Solver Foundation (MSF).

You can use MSF for creating a wide range of different types of optimization models in C# for example. From there, you can link your solver model back to Excel with Excel-DNA http://exceldna.codeplex.com/. This means, that you can set up your data and other parameters in Excel workbook, then use C# and MSF to perform desired optimization routine without interacting with Excel and finally, push the results back to Excel worksheet. In other words, you can create Excel UDF for solving optimization routines.

Let us first go through, what we are going to reach in this post:
  1. We set up MSF to be used in Visual C# 2010 Express.
  2. We create optimization model in C# by using MSF.
  3. We set up Excel-DNA linking for C# program.
  4. We create interface function for handling data between Excel and C# program.
  5. We verify MSF model results against Excel Frontline Solver results.
So, let us get into it. First, open your Visual C# 2010 Express.

Microsoft Solver Foundation setup

First, we have to find MSF dll library file. We need to have Solver Foundation v3.0 - DLL only. I found it from here
http://archive.msdn.microsoft.com/solverfoundation/Release/ProjectReleases.aspx?ReleaseId=1799

When you have that dll file, we create a new C# Class Project in your Visual C# 2010 Express (File - New Project - Class Library - Name=MSF_Solver). Remember to save your project. Now, create reference to that MSF dll library file (Solution Explorer - References - Add Reference - Browse). If everything goes correctly, you should be able to build the following test program without any errors. This program is just for quick testing, that your MSF is ready for servicing.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Microsoft.SolverFoundation.Services;

namespace MSF_Solver
{
    public class ExcelFunctions
    {
        SolverContext solver = SolverContext.GetContext();
    }
}

I assume that everything went correctly without any errors in your build. At this stage, we have set up MSF to be used in our Visual Studio. One word about the tools: I am using Visual C# 2010 Express and having .NET Framework 4.0 as my Target framework.

The C# Program

Now we need to create the actual program, which will be using MSF to perform desired optimization routine. First, we create ISolver interface, from which all possible Solver models are going to be implemented. This interface is having only one method (solve) and it returns Dictionary data structure (string, double). In your Visual Studio, add new Interface to your project (Solution Explorer - MSF_Solver Project - Add - Add New Item - Interface - Name=ISolver). You can copy-paste the following program into this interface.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace MSF_Solver
{
    public interface ISolver
    {
        Dictionary<string, double> solve();
    }
}

Next, we need to create an implementation for this interface. Add a new Class to your project (Solution Explorer - MSF_Solver Project - Add - Add New Item - Interface - Name=PolynomialFitting). You can copy-paste the following program into this class.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Microsoft.SolverFoundation.Services;
//
namespace MSF_Solver
{
    //
    // PolynomialFitting implementation for ISolver interface is fitting nth degree polynomial 
    // approximation function to a given set of data points by using ordinary least squares method
    public class PolynomialFitting : ISolver
    {
        private Dictionary<double, double> curve; // data points to be fitted (x-value, y-value)
        private int degree; // define degree for polynomial function
        //
        public PolynomialFitting(Dictionary<double, double> curve, int degree)
        {
            this.curve = curve;
            this.degree = degree;
        }
        //
        // ISolver interface method implementation
        public Dictionary<string, double> solve()
        {
            // create solver model
            SolverContext solver = SolverContext.GetContext();
            solver.ClearModel();
            Model model = solver.CreateModel();
            //
            // create decision variables for solver model, based on given polynomial degree
            Dictionary<string, double> coefficients = new Dictionary<string, double>();
            for (int i = 0; i <= degree; i++)
            {
                coefficients.Add("b" + i.ToString(), 0.0);
            }
            //
            // add decision variables to solver model
            var decision = coefficients.Select(it => new Decision(Domain.Real, it.Key));
            model.AddDecisions(decision.ToArray());
            //
            // create objective function by using SumTermBuilder
            var objectiveFunction = new SumTermBuilder(curve.Count);
            //
            // itarate through all given data points
            foreach (KeyValuePair<double, double> kvp in curve)
            {
                // create term object incrementally for squared error (y estimate - y actual)^2
                Term subTerm = 0;
                for (int i = 0; i < model.Decisions.Count(); i++)
                {
                    // y-estimate = (b0 * maturity^0 + b1 * maturity^1 + ... + bn * maturity^n)
                    subTerm += model.Decisions.ElementAt(i) * Math.Pow(kvp.Key, i);
                }
                subTerm -= kvp.Value; //  subterm = (y estimate - y actual) = e
                subTerm *= subTerm; // subterm = e^2
                objectiveFunction.Add(subTerm); // add subterm to objectiveFunction (SumTermBuilder)
            }
            //
            // define optimization type and give objective function SUM(e^2) to be minimized
            model.AddGoal("SumOfSquaredErrors", GoalKind.Minimize, objectiveFunction.ToTerm());
            //
            // solve model and transfer results (optimized decision variables) from 
            // model into a dictionary object which will be returned for the caller
            Solution solution = solver.Solve();
            Dictionary<string, double> parameters = new Dictionary<string, double>();
            foreach (Decision parameter in model.Decisions)
            {
                parameters.Add(parameter.Name, parameter.ToDouble());
            }
            return parameters;
        }
    }
}

The previous class takes in a given set of data points (x, y) inside Dictionary data structure, along with the information about the required polynomial degree of the approximation function to be fitted with data. Relevant comments are included inside the program. More information on MSF can be found with Google. Also, Here are two excellent hands-on examples on using MSF from Mathias Brandewinder: 
http://www.clear-lines.com/blog/post/First-steps-with-the-Microsoft-Solver-Foundation.aspx http://www.clear-lines.com/blog/post/Create-optimization-programs-dynamically-with-C-and-the-Microsoft-Solver-Foundation.aspx

At this point, we have created our desired optimization model in C# by using MSF. Next, we need to set up Excel-DNA for linking our C# program with Excel Worksheet.

Excel-DNA part

First we have to find Excel-DNA files. I was able to find these from here http://exceldna.codeplex.com/ by going to Downloads. I have been using ExcelDna-0.29 so far, but there is now version 0.30 available, as I am writing this post. Download zip file and unzip the content into some appropriate folder.

Next, in our C# program, we need to create reference to Excel-DNA dll file (Solution Explorer - References - Add Reference - Browse). You should find ExcelDna.Integration.dll file. In Properties window, mark this reference as Copy Local = False.

Next, Add a new file (dna file) to the project (Solution Explorer - MSF_Solver Project - Add - Add New Item - Text File - Name=MSF_Solver.dna) with the following content:

<DnaLibrary Name="MSF_Solver" RuntimeVersion="v4.0">
  <ExternalLibrary Path="MSF_Solver.dll" />
</DnaLibrary>

In the properties for this file, set that this file will be copied to the Output directory. Set Copy to Output Directory = Copy if newer.

Then, create a copy of the ExcelDna.xll file (you can find this xll file in your unzipped Excel-Dna folder) into the project directory, and rename it to be MSF_Solver.xll. Add this file to the project (Solution Explorer - MSF_Solver Project - Add - Add Existing Item - MSF_Solver.xll), and in the properties for this file, ensure that it will be copied to the Output directory. Set Copy to Output Directory = Copy if newer.

After these important settings, build your solution. I assume everything has gone well to this point. Setting up Excel-DNA is not difficult, but still one needs to be very careful. One slip in settings and you will spend a lot of time with your compiler, wondering why this is not working. So, be careful out there.

Detailed instructions for setting up Excel-DNA functionalities in Visual Studio can be found from here http://exceldna.codeplex.com/documentationFind the following Word documentation Excel-DNA - Step-by-step C# add-in.doc In that documentation, find the section Creating a new add-in library in C#. The process of setting up Excel-DNA has been described in a very detailed way in that section, along with some screen captures.

At this point, we have set up Excel-DNA files for linking our C# program with Excel Workbook. Next, we are going to create interface function for handling data between Excel and C# program.

Interface function between Excel and C#

At this point, you should have that one Class (ExcelFunctions) existing in your current project (the one, in which we tested MSF originally). You can copy-paste the following code into that class.



using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using ExcelDna.Integration;

namespace MSF_Solver
{
    public class ExcelFunctions
    {
        [ExcelFunction(Category="Excel-DNA functions", Description = "Coefficients for nth degree polynomial curve fit to a given set of data points by using OLS method")]
        public static object[,] PolynomialCurveFitting([ExcelArgument(Description = "2-dimensional array of (x, y) values")] Object[,] curve, [ExcelArgument(Description = "Degree for polynomial function")] int degree)
        {
            // number of data points for a given curve is dynamic
            int rows = curve.GetUpperBound(0);
            //
            // read a given curve data into a dictionary
            Dictionary<double, double> dict = new Dictionary<double, double>();
            for (int i = 0; i <= rows; i++)
            {
                dict.Add((double)curve.GetValue(i, 0), (double)curve.GetValue(i, 1));
            }
            //
            // create appropriate solver model, perform curve fitting and receive optimized coefficients
            ISolver solver = new PolynomialFitting(dict, degree);
            Dictionary<string, double> coefficients = solver.solve();
            //
            // create array for output
            // key is coefficient name (string) and value is coefficient value (double)
            object[,] arr = new object[coefficients.Count , 2];
            //
            for (int i = 0; i < coefficients.Count; i++)
            {
                arr[i, 0] = (string)coefficients.ElementAt(i).Key;
                arr[i, 1] = (double)coefficients.ElementAt(i).Value;
            }
            return arr;
        }
    }
}

First, PolynomialCurveFitting method is receiving data as 2-dimensional object array from Excel. The program then reads that data into a Dictionary data structure. This Dictionary is a feed for ISolver implementation (solver), which performs the optimization part and returns optimized coefficients inside a new Dictionary back to this method (coefficients). Finally, the content of this result Dictionary is read into a new object array (arr) and returned back to Excel.

Now, build your fully completed project. At this stage, you should have the following four files in your Project directory (MSF_Solver\bin\Debug):
  1. MSF_Solver.dll
  2. MSF_Solver.dna
  3. MSF_Solver.pdb
  4. MSF_Solver.xll
Program test run - curve fitting

We test our program and verify our MSF model results against Frontline Solver results. First, set up the following swap curve data into a new Excel workbook.

0,08 0,19
0,25 0,27
0,5 0,29
1 0,35
2 0,51
5 1,38
10 2,46
20 3,17
30 3,32

While your Excel is open, go to your Project folder (MSF_Solver\bin\Debug) and double-click MSF_Solver.xll file. You may not notice anything, but behind the scenes your xll file is loaded into your Excel Workbook. If something has gone wrong, you will get an error message at this point.

Now, let us perform a curve fitting for that data with second degree polynomial function. We are going to have three output coefficients from the program (b0, b1, b2). Our C# method is returning the name of the coefficient and its optimized value. All in all, we are going to receive 3x2 output variant array in this example case. So, when using PolynomialCurveFitting function in Excel, remember to select first 3x2 array in your worksheet before creating your function formula and then press CTRL+SHIFT+ENTER to get the result array from C#.

Below here is my test run results. I have been testing this program also for other degree polynomial approximations and it seems to be working well at least against Excel Solver. We could make preliminary conclusion, that our C# MSF solver model is working as required.
MSF solver Excel solver
b0 0,1434 b0 0,1434
b1 0,2684 b1 0,2684
b2 -0,0055 b2 -0,0055

Before publishing this post, I have been following all my instructions described on this posting, re-created this program and used PolynomialCurveFitting function in Excel successfully two times. So, if you follow all the instructions carefully, you should get the working program.

Some Afterthoughts

Let us face one fact: people in banks are accustomed with Excel and they want to use it. Excel is - and probably will be for a long time - the daily workhorse. Tools like Excel-DNA can open up a lot of new doors what we can do in Excel behind the scenes. Personally I find this approach of creating a program completely outside Excel (VBA) and linking that program back to be used in Excel, to be a wonderful tool. It feels a bit like using new Rolls Royce engine in your old Volkswagen.

My personal Thank You this time goes directly to Govert Van Drimmelen, who AFAIK has developed Excel-DNA pretty much on his own. What this world really would be without such innovators? Thank You also for Mathias Brandewinder and his clear-lines.com blog for those two excellent articles on MSF. Also, Thank You for MSF team for developing such a great optimization tool.

And finally, Thank You for reading this blog again. I really hope that you have got something out from this. Have a great midsummer day!
-Mike