AlgLib : Ho-Lee Calibration Using Levenberg-Marquardt algorithm in VBA

Some time ago, I published one possible C# implementation for Ho-Lee one-factor model calibration scheme using AlgLib numerical libraries. This time I will present an implementation for the same scheme in VBA. General information concerning Levenberg-Marquardt algorithm implementation in AlgLib has been presented in here. Libraries for VBA (collection of BAS module files, which are going to be included into VBA project) can be downloaded from here.

A few words about this implementation. AlgLibLMASolver class uses AlgLib library functions (functions from 21 different modules) for processing (creating optimization model, setting conditions, processing iterations). One data member within this class is having a type of IModel. This data member is actually a reference to an interface, which provides a set of functions for all required calculations (objective function value, values for function terms, partial derivative values for function terms). Since all possible implementations for any interface method must honor signatures exactly, there is a problem with VBA since it does not have a real constructor mechanism. I have chewed this issue in here. It might help to explain the reason, why I have been distributing input parameters for interface implementation by using Dictionary object. Finally, HoLeeZeroCouponCalibration class is implementing IModel interface (a set of functions for all required calculations). In essence, algorithms (AlgLib-related processing) and data (values calculated specifically by using Ho-Lee model) have been completely separated. Needless to say, this type of scheme is flexible for new implementations.

Create a new VBA project and copy-paste the following classes and modules into this project. Also, import all required 21 AlgLib BAS files into this project.

' CLASS : AlgLibLMASolver
Option Explicit
'
' The following 21 AlgLib modules are required for succesfull compilation of this project :
' ablas, ablasf, ap, bdsvd, blas, creflections, densesolver, hblas, linmin, matinv,
' minlbfgs, minlm, ortfac, rcond, reflections, rotations, safesolve, sblas, svd, trfac, xblas
'
Private state As MinLMState
Private report As MinLMReport
Private n As Long
Private m As Long
Private x() As Double
Private model As IModel
Private epsF As Double
Private epsG As Double
Private epsX As Double
Private iterations As Long
'
Public Function initialize( _
    ByVal numberOfVariables As Long, _
    ByVal numberOfEquations As Long, _
    ByRef changingVariables() As Double, _
    ByRef callbackModel As IModel, _
    ByVal epsilonF As Double, _
    ByVal epsilonG As Double, _
    ByVal epsilonX As Double, _
    ByVal maximumIterations As Long)
    '
    n = numberOfVariables
    m = numberOfEquations
    x = changingVariables
    Set model = callbackModel
    epsF = epsilonF
    epsG = epsilonG
    epsX = epsilonX
    iterations = maximumIterations
End Function
'
Public Sub Solve()
    '
    ' create solver scheme using functions and analytical partial derivatives
    Call MinLMCreateFJ(n, m, x, state)
    ' set stopping conditions
    Call MinLMSetCond(state, epsG, epsF, epsX, iterations)
    '
    ' process iterations
    Do While MinLMIteration(state)
        '
        ' calculate value for objective function
        If (state.NeedF) Then
            '
            model.callBackObjectiveFunction state
        End If
        '
        ' calculate values for functions and partial derivatives
        If (state.NeedFiJ) Then
            '
            model.callBackFunction state
            model.callBackJacobian state
        End If
    Loop
    '
    ' process results
    Call MinLMResults(state, x, report)
End Sub
'
' public accessor to (MinLMState) state
Public Property Get GetState() As MinLMState
    GetState = state
End Property
'
' public accessor to (MinLMReport) report
Public Property Get GetReport() As MinLMReport
    GetReport = report
End Property
'
' public accessor to hard-coded report
Public Property Get GetPrettyPrintReport() As String
    '
    Dim message As String
    message = "*** AlgLibLMASolver execution report " + VBA.CStr(VBA.Now) + " ***" + VBA.vbNewLine
    message = message + "TerminationType : " + VBA.CStr(report.TerminationType) + VBA.vbNewLine
    message = message + "Iterations : " + VBA.CStr(report.IterationsCount) + VBA.vbNewLine
    message = message + "Objective function : " + VBA.CStr(state.f) + VBA.vbNewLine
    message = message + VBA.vbNewLine
    '
    Dim i As Integer
    For i = 0 To (state.n - 1)
        message = message + "x(" + VBA.CStr(i) + ") = " + VBA.CStr(state.x(i)) + VBA.vbNewLine
    Next i
    '
    GetPrettyPrintReport = message
End Property
'
'
'
'
'
' CLASS : IModel
Option Explicit
' set of functions for IModel interface
Public Function initialize(ByRef parameters As Scripting.Dictionary)
    ' assign required member data wrapped into dictionary
End Function
'
Public Function callBackObjectiveFunction(ByRef state As MinLMState)
    ' calculate objective function value
End Function
'
Public Function callBackFunction(ByRef state As MinLMState)
    ' calculate values for (non-squared) function terms
End Function
'
Public Function callBackJacobian(ByRef state As MinLMState)
    ' calculate partial derivative values for (non-squared) function terms
End Function
'
'
'
'
'
' CLASS : HoLeeZeroCouponCalibration
Option Explicit
'
Implements IModel
'
Private s As Double
Private r As Double
Private t() As Double
Private z() As Double
'
Private Function IModel_initialize(ByRef parameters As Scripting.IDictionary)
    '
    s = parameters(HOLEE_PARAMETERS.sigma)
    r = parameters(HOLEE_PARAMETERS.shortRate)
    t = parameters(HOLEE_PARAMETERS.maturity)
    z = parameters(HOLEE_PARAMETERS.zeroCouponBond)
End Function
'
Private Function IModel_callBackObjectiveFunction(ByRef state As MinLMState)
    '
    ' calculate value for aggregate objective function
    Dim i As Integer
    Dim hoLeeZero As Double
    Dim f As Double: f = 0
    '
    ' loop through number of equations
    For i = 0 To (state.m - 1)
        '
        hoLeeZero = VBA.Exp(-(1 / 2) * state.x(i) * (t(i) ^ 2) + (1 / 6) * (s ^ 2) * (t(i) ^ 3) - r * t(i))
        f = f + (z(i) - hoLeeZero) ^ 2
    Next i
    state.f = f
End Function
'
Private Function IModel_callBackFunction(ByRef state As MinLMState)
    '
    ' calculate values for (non-squared) function terms
    Dim i As Integer
    Dim hoLeeZero As Double
    '
    ' loop through number of equations
    For i = 0 To (state.m - 1)
        '
        hoLeeZero = VBA.Exp(-(1 / 2) * state.x(i) * (t(i) ^ 2) + (1 / 6) * (s ^ 2) * (t(i) ^ 3) - r * t(i))
        state.FI(i) = (z(i) - hoLeeZero)
    Next i
End Function
'
Private Function IModel_callBackJacobian(ByRef state As MinLMState)
    '
    ' calculate partial derivative values for (non-squared) function terms
    Dim i As Integer, J As Integer
    Dim hoLeeZero As Double
    '
    ' 1. individual (non-squared) function terms
    ' loop through number of equations
    For i = 0 To (state.m - 1)
        '
        hoLeeZero = VBA.Exp(-(1 / 2) * state.x(i) * (t(i) ^ 2) + (1 / 6) * (s ^ 2) * (t(i) ^ 3) - r * t(i))
        state.FI(i) = (z(i) - hoLeeZero)
    Next i
    '
    ' 2. partial derivatives for all (non-squared) function terms
    ' loop through number of equations
    For i = 0 To (state.m - 1)
        '
    ' loop through number of variables
        For J = 0 To (state.n - 1)
            '
            Dim derivative As Double: derivative = 0
            ' partial derivative is non-zero only for diagonal cases
            If (i = J) Then
                derivative = (1 / 2) * VBA.Exp(1) * t(J) ^ 2
                state.J(i, J) = derivative
            End If
        Next J
    Next i
End Function
'
'
'
'
'
' MODULE : DataStructures
Option Explicit
'
Public Enum HOLEE_PARAMETERS
    sigma = 1
    shortRate = 2
    maturity = 3
    zeroCouponBond = 4
End Enum
'
'
'
'
'
' TESTER MODULE
Option Explicit
'
' Ho-Lee model calibration example
Public Sub AlglibTester()
    '
    ' MODEL part
    ' construct all required inputs and model to be calibrated
    Dim sigma As Double: sigma = 0.00039
    Dim shortRate As Double: shortRate = 0.00154
    '
    Dim maturity(0 To 9) As Double
    maturity(0) = 1: maturity(1) = 2: maturity(2) = 3: maturity(3) = 4: maturity(4) = 5:
    maturity(5) = 6: maturity(6) = 7: maturity(7) = 8: maturity(8) = 9: maturity(9) = 10
    '
    Dim zero(0 To 9) As Double
    zero(0) = 0.9964: zero(1) = 0.9838: zero(2) = 0.9611: zero(3) = 0.9344: zero(4) = 0.9059:
    zero(5) = 0.8769: zero(6) = 0.8478: zero(7) = 0.8189: zero(8) = 0.7905: zero(9) = 0.7626
    '
    ' assign parameters into dictionary wrapper
    Dim parameters As New Scripting.Dictionary
    parameters.Add HOLEE_PARAMETERS.sigma, sigma
    parameters.Add HOLEE_PARAMETERS.shortRate, shortRate
    parameters.Add HOLEE_PARAMETERS.maturity, maturity
    parameters.Add HOLEE_PARAMETERS.zeroCouponBond, zero
    '
    ' create and initialize calibration model
    Dim model As IModel: Set model = New HoLeeZeroCouponCalibration
    model.initialize parameters
    '
    ' SOLVER part
    Dim Theta(0 To 9) As Double ' assign initial guesses
    Theta(0) = 0.001: Theta(1) = 0.001: Theta(2) = 0.001: Theta(3) = 0.001: Theta(4) = 0.001:
    Theta(5) = 0.001: Theta(6) = 0.001: Theta(7) = 0.001: Theta(8) = 0.001: Theta(9) = 0.001
    '
    Dim numberOfVariables As Long: numberOfVariables = 10
    Dim numberOfEquations As Long: numberOfEquations = 10
    Dim epsilonF As Double: epsilonF = 0.000000000001
    Dim epsilonG As Double: epsilonG = 0.000000000001
    Dim epsilonX As Double: epsilonX = 0.000000000001
    Dim maximumIterations As Long: maximumIterations = 25000
    '
    ' create and initialize solver model
    Dim solver As New AlgLibLMASolver
    solver.initialize _
        numberOfVariables, _
        numberOfEquations, _
        Theta, _
        model, _
        epsilonF, _
        epsilonG, _
        epsilonX, _
        maximumIterations
    '
    ' solve calibration model
    solver.Solve
    '
    ' print hard-coded report containing values for
    ' objective function, variables and other information
    Debug.Print solver.GetPrettyPrintReport
End Sub
'

The results from this calibration model have been verified against the previous results.

Importing several files into project may involve considerable amount of cruel and unusual repetitive labour. For this specific reason, I have also been woodshedding a separate module (employing VBIDE object), which might give some relief when babysitting those AlgLib modules.

' The following dll libraries need to be referenced :
' Microsoft Visual Basic for Applications Extensibility 5.X, Microsoft Scripting Runtime
Option Explicit
Option Base 0
'
' address to a list, which contains all BAS files which will be included into project
Const listFolderPathName As String = "C:\AlgLib\vba\AlgLibLMAModules.txt"
' address to a folder, which contains all AlgLib BAS files
Const moduleFolderPathName  As String = "C:\AlgLib\vba\alglib-2.6.0.vb6\vb6\src\"
' select TRUE, if Require Variable Declaration in editor is tagged
Const removeOptionExplicitDirective As Boolean = True
'
Public Sub ImportModules()
    '
    ' create a list of modules to be imported into this project
    Dim list() As String: list = createProjectModuleList
    ' import modules into active project
    import list
End Sub
'
Public Sub ExportModules()
    '
    ' create a list of modules to be exported from this project
    Dim list() As String: list = createProjectModuleList
    ' export modules from active project into a defined folder
    export list
End Sub
'
Public Sub RemoveModules()
    '
    ' create a list of modules to be removed from this project
    Dim list() As String: list = createProjectModuleList
    ' delete modules from active project
    remove list
End Sub
'
Private Function import(ByRef list() As String)
    '
    Dim editor As VBIDE.VBProject
    Set editor = ActiveWorkbook.VBProject
    Dim fileSystem As Scripting.FileSystemObject: Set fileSystem = New Scripting.FileSystemObject
    '
    ' loop through all files in a specific source folder for modules
    Dim filesInGivenList As Integer: filesInGivenList = UBound(list) + 1
    If (filesInGivenList = 0) Then Exit Function
    '
    Dim module As VBIDE.VBComponent
    Dim file As Scripting.file
    For Each file In fileSystem.GetFolder(moduleFolderPathName).Files
        '
        ' if there is a given list of specific files to be included
        ' select only the files in that list to be imported into project
        If Not (moduleIsIncluded(file.Name, list)) Then GoTo skipPoint
        '
        Set module = editor.VBComponents.Add(vbext_ct_StdModule)
        If (removeOptionExplicitDirective) Then module.CodeModule.DeleteLines 1
        module.Name = VBA.Split(file.Name, ".")(0)
        module.CodeModule.AddFromFile file.Path
skipPoint:
    Next
End Function
'
Private Function export(ByRef list() As String)
    '
    Dim filesInGivenList As Integer: filesInGivenList = UBound(list) + 1
    If (filesInGivenList = 0) Then Exit Function
    '
    Dim editor As VBIDE.VBProject
    Set editor = ActiveWorkbook.VBProject
    Dim module As VBIDE.VBComponent
    '
    ' loop through all modules
    For Each module In editor.VBComponents
        '
        ' export module only if it is included in the list
        If (moduleIsIncluded(module.Name + ".bas", list)) Then
            module.export moduleFolderPathName + module.Name + ".bas"
        End If
    Next
End Function
'
Private Function remove(ByRef list() As String)
    '
    Dim filesInGivenList As Integer: filesInGivenList = UBound(list) + 1
    If (filesInGivenList = 0) Then Exit Function
    '
    Dim editor As VBIDE.VBProject
    Set editor = ActiveWorkbook.VBProject
    Dim module As VBIDE.VBComponent
    '
    ' loop through all modules
    For Each module In editor.VBComponents
        '
        ' remove module only if it is included in the list
        If (moduleIsIncluded(module.Name + ".bas", list)) Then
            module.Collection.remove module
        End If
    Next
End Function
'
Private Function moduleIsIncluded(ByVal FileName As String, ByRef list() As String) As Boolean
    '
    ' check if a given file name is in the list
    Dim isIncluded As Boolean: isIncluded = False
    Dim i As Integer
    For i = 0 To UBound(list)
        If (FileName = list(i)) Then
            isIncluded = True
            Exit For
        End If
    Next i
    moduleIsIncluded = isIncluded
End Function
'
Private Function createProjectModuleList() As String()
    '
    ' create a list of file names from text file
    Dim fileSystem As Scripting.FileSystemObject: Set fileSystem = New Scripting.FileSystemObject
    Dim fileReader As Scripting.TextStream: Set fileReader = fileSystem.OpenTextFile(listFolderPathName, ForReading)
    Dim fileStreams As String: fileStreams = fileReader.ReadAll
    Dim streams As Variant: streams = VBA.Split(fileStreams, VBA.vbNewLine)
    Dim list() As String: ReDim list(0 To UBound(streams))
    Dim i As Integer
    For i = 0 To UBound(streams)
        list(i) = VBA.Trim(streams(i))
    Next i
    createProjectModuleList = list
End Function
'

Finally, thanks for reading this blog.
-Mike

Excel/VBA : Publishing HTML Reports from Excel

In my opinion, creating reports directly from Excel workbook is a bit old-fashioned way to handle the case. For any official reporting (meant to be published for a wide audience of stakeholders, needs to have traceable history), my recommendation is pretty much the following : process calculations (say, by using third-party software), process results into SQL server database (to store report history), create views using relevant database tables, finally use SSRS by linking report graphical objects with database views for report publishing.

However, there might still be some cases, in which this kind of processing would definitely be considered as overkill. Say, a batch of processed daily calculation results must be shared within the unit for some internal (informal) checking. For such cases, creating report directly from Excel workbook by using publishObject in VBA is quite handy and inexpensive solution. So, what to do then ? First, create a report page (one centralized worksheet) from which all the reports (there can be more than just one report to be published) will be published. For each report, define a named range which covers the exact area to be published. As an example, the screenshot below shows imaginary Excel report what I have created. Note, that the actual file name for HTML file has been defined as a comment property of Excel.Name object. Note also, that the figures in table below are arbitrary.

Next, I have defined a network folder (hard-coded in the program) into which this report will be published as html page. Finally, I have implemented the program (presented below) in Excel, from which the report (or several reports) will be published. As a user will press command button (create reports), the program will create html file, as shown in the screenshot below.

VBA program for handling command button "click" event is presented below.

Option Explicit
'
Private Const folderPathNameForHTMLFiles As String = "C:\temp\HTMLReporting\"
'
Private Sub btnCreateReports_Click()
    '
    ' deactivate command button pressing
    ActiveSheet.Range("A1").Select
    '
    Dim reportRangeName As Excel.Name
    Dim reportRange As Excel.Range
    Dim report As PublishObject
    Dim reports As PublishObjects: Set reports = ActiveWorkbook.PublishObjects
    '
    ' loop through all names within active workbook
    For Each reportRangeName In ActiveWorkbook.Names
        '
        ' preparation for the case, in which the name is corrupted
        On Error GoTo exitPoint
        ' handle only the names related to ranges in reporting worksheet
        If (reportRangeName.RefersToRange.Worksheet.Name = ActiveSheet.Name) Then
            '
            ' disable screen updating
            Excel.Application.ScreenUpdating = False
            ' create Excel range for report
            Set reportRange = Worksheets(ActiveSheet.Name).Range(reportRangeName.Name)
            reportRange.Select
            '
            ' create path and name for HTML report file
            ' file name (including html extension) is defined as comment within Excel name
            Dim HTMLFilePathName As String
            HTMLFilePathName = folderPathNameForHTMLFiles + reportRangeName.Comment
            '
            ' create and publish report as publishedObject object
            Set report = reports.Add( _
                SourceType:=xlSourceRange, _
                fileName:=HTMLFilePathName, _
                Sheet:=reportRange.Worksheet.Name, _
                Source:=reportRange.Address, _
                HtmlType:=xlHtmlStatic)
            report.Publish True
        End If
        '
exitPoint:
    Next
    '
    ' finally, delete all reports from active workbook
    reports.Delete
    Range("A1").Select
    Excel.Application.ScreenUpdating = True
End Sub
'

Finally, thanks a lot for reading this blog.
-Mike

Batch scripting : managing configurational complexity

Processing calculations using third-party analytical software may turn out to be more like a process, requiring several independent tasks to be executed in a certain order, before any actual magic will happen. The one I have been using, was initially delivered with relatively large collection of black box executables. I decided to streamline those executables out from the picture by (first studying and then) using API. However, there were still a lot of tricky housekeeping stuff to be done before and after the actual number crunching (ex. required source data creation, result processing).

For all required housekeeping stuff, I created independent programs (executables), which were wrapped inside batch script files to be called, since a lot of auxiliary information was required as feed for executables (ex. address to configuration file). The process was working like a dream up to the point, in which some minor changes were made into existing network folder structure. I confess the following : almost all path addresses in batch script wrapper files were actually hard-coded. In order to compensate this horrible tragedy, I decided to use relative paths in script files. Solution was again technically working well, but script files were quickly being polluted by lines of incomprehensible streams. Solving possible issues would definitely be excrutiating experience for any non-experienced person. I also realized, that given some specific change to be made in the future, there would still be janitor work to be done with several script files. Bad omen.

As being non-expert on batch scripting, I finally decided to go to direction which is the most comfortable for me : constructing all required configurations from one centralized place. For this specific process, I created a batch script which is creating all possible configuration strings (mostly addresses to folders containing other configurations or executables). This script would then be called, before any other batch script content would be executed. Since the law of conservation of misery states "total amount of misery in a system is constant", one may well then ask what was the actual gain ? Firstly, lines of commands in wrapper script files were much more comprehensible (think about someone else on your desk when something happens) and secondly, given any future changes in network directory, changes would be made only in one place. At least for me, these issues are more than justifying the proposed scheme.

Batch script implementations

Script for creating "global" configurations

Script for using "global" configurations script

Script (MarketDataFeeder.bat) being called inside previous script

Test run (executing Datafeed.bat)

Finally, thanks for reading this blog.
-Mike