Python: Implementing Factory Method Design Pattern

Ideally, program should be closed for modifications, but open for extensions and hard-coded stuff should be avoided like plague. This is the point, where Design Patterns are usually stepping into picture. When the code base will grow enough in size and we are in charge of production-level programs, it is relatively easy to justify the usage of patterns. In this post, Python mutation of Factory Method Pattern is introduced for creating Task class objects, based on argument feed received from different types of sources. Complete program can be downloaded from my github page.

Imaginary scheme

Third-party analytical software is used for processing different types of complex calculations. One calculation request requires a set of specific arguments from the client program. Moreover, different types of requests will require different set of arguments (number of arguments, types of arguments). Before sending calculation request for analytical software, client program must collect arguments for all tasks to be processed from somewhere and we would like to leave the source for such task arguments open (json, text, console, xml, etc.).

Json source file

{
  "taskName": "ANZ.XVA",
  "taskType": "XVA",
  "paths": 1000,
  "storeExposures": "True" 
}

Text source file

taskName,RBC.PV
taskType,PV

Program

import sys, json, csv

# class for hosting calculation task-related information
# NOTE: the actual method for creating 'calculation request' is currently not implemented
class Task(object):
    def __init__(self, arguments):
        self.arguments = arguments
        
    # non-relevant dummy method, which just writes out given arguments
    def print(self):
        for k, v in self.arguments.items():
            print(k, 'is', v)
        
    @classmethod
    # factory: create task object from json file
    def from_json(cls, path)->'Task':
        arguments = json.load(open(path, 'r'))
        return cls(arguments)
    
    @classmethod
    # factory: create task object from text file
    def from_text(cls, path)->'Task':
        arguments = dict((k, v) for k, v in csv.reader(open(path, 'r')))
        return cls(arguments)
    
    @classmethod
    # factory: create task object from console input
    def from_console(cls)->'Task':
        arguments = json.loads(input())
        return cls(arguments)        

path = '/home/mikejuniperhill/json.feed/task.json'
task = Task.from_json(path)
task.print()

path = '/home/mikejuniperhill/json.feed/task.txt'
task = Task.from_text(path)
task.print()

task = Task.from_console()
# console feed string: {"taskName":"ANZ.WHATIF.XVA","taskType":"WHATIF","referenceTask":"ANZ.XVA"}
task.print()

Program output

Thanks for reading this blog.
-Mike

QuantLib-Python: Transaction Builder

Even the fanciest restaurant needs to have dedicated staff to visit fresh market and peel potatoes, in order to have everything ready for head chef to prepare delicious late night dinners for us. Similarly, having amazing analytics library (such as QuantLib) available for calculations is still only "halfway home", since all required data inputs (ex. transactions) need to be constructed beforehand. It is highly preferred for this part of the process to be performed in a way, that maximum re-configurability would be maintained, while manual labour would be completely avoided.

In a nutshell, this post will present, how to go from having several XML configuration files for specific QuantLib transaction in a directory (shown on the left side), to have all these transactions constructed and processed through QuantLib (shown on the right side).

Implement the following Python program. First, it will create QuantLib flat yield term structure and discounting bond pricing engine, request a batch of constructed QuantLib transactions from TransactionManager method (located in a separate QuantLibTransactionBuilder), then assign pricing engine for each transaction and finally, print calculated NPV along with some other transaction-related information.

%config IPCompleter.greedy = True
from QuantLib import *
from QuantLibTransactionBuilder import *

# create valuation curve and pricing engine
tradeDate = Date(28, November, 2018)
settlementDate = TARGET().advance(tradeDate, Period(2, Days))
curveHandle = YieldTermStructureHandle(FlatForward(settlementDate, 0.01, Actual360()))
engine = DiscountingBondEngine(curveHandle)

# create QuantLib transactions from repository XML files
transactionsFolder = '/home/TransactionRepository/'
transactions = TransactionManager(transactionsFolder)

# set pricing engine for all transactions, request PV
for t in range(len(transactions)):
    transactions[t].setPricingEngine(engine)
    print('Maturity: ' + str(transactions[t].maturityDate()))
    print('Notional: ' + str(transactions[t].notional()))
    print('NPV: ' + str(transactions[t].NPV()))
    print()

Next, implement the following new module (QuantLibTransactionBuilder.py), which contains specific transaction builder for QuantLib ZeroCouponBond instrument, static class methods for QuantLib-related type conversions and one method for handling a set of given XML files and their conversions into QuantLib objects. Given comments in this module should give crystal clear picture what is happening here.

import os
import importlib
import re as Regex
from bs4 import BeautifulSoup # sudo apt-get install python3-bs4
from QuantLib import *

# static class for conversions from string to QuantLib objects/enumerators
class QL():
    @staticmethod
    def to_date(s):
        monthDictionary = {
            '01': January, '02': February, '03': March,
            '04': April, '05': May, '06': June,
            '07': July, '08': August, '09': September,
            '1': January, '2': February, '3': March,
            '4': April, '5': May, '6': June,
            '7': July, '8': August, '9': September,
            '10': October, '11': November, '12': December
        }
        arr = Regex.findall(r"[\w']+", s)
        day = int(arr[2])
        month = monthDictionary[arr[1]]
        year = int(arr[0])
        return Date(day, month, year)
    @staticmethod
    def to_businessDayConvention(s):
        if (s.upper() == 'MODIFIEDFOLLOWING'): return ModifiedFollowing
        # add new businessdayconvention here
    @staticmethod
    def to_calendar(s):
        if (s.upper() == 'TARGET'): return TARGET()
        # add new calendar here


# loop through XML files in a given folder
# for each transaction, read 'transactionBuilder' attribute from XML data
# call configured transaction builder, build transaction and add it to list
# finally, return list of all built QuantLib transactions to client 
def TransactionManager(folder):
    transactions = []
    files = os.listdir(folder)
    for f in range(len(files)):
        soup = BeautifulSoup(open(folder + files[f]).read(), 'xml')
        transactionBuilder = soup.find('transactionBuilder').get_text()
        builder = getattr(importlib.import_module('QuantLibTransactionBuilder'), transactionBuilder)(folder + files[f])
        transaction = builder
        transactions.append(transaction)
    return transactions


# method for constructing QuantLib 'ZeroCouponBond' object from a given XML data 
def ZeroCouponBondBuilder(filePathName):
    # assign transaction data from XML to variables using soup
    # use conversion class methods for all QuantLib-related conversion 
    soup = BeautifulSoup(open(filePathName).read(), 'xml')
    tradeDate = QL.to_date(soup.find('tradeDate').get_text())
    settlementDate = QL.to_date(soup.find('settlementDate').get_text())
    calendar = QL.to_calendar(soup.find('calendar').get_text())
    faceAmount = float(soup.find('faceAmount').get_text())
    maturityDate = QL.to_date(soup.find('maturityDate').get_text())
    paymentConvention = QL.to_businessDayConvention(soup.find('paymentConvention').get_text())
    settlementDays = settlementDate - tradeDate 
    # create QuantLib object by calling appropriate constructor
    return ZeroCouponBond(settlementDays, calendar, faceAmount, maturityDate, paymentConvention)

All desired new transaction builders should be implemented into previous module by adding another builder method. Also, corresponding builder method name should be also set into transaction XML file as one input parameter.

<ZeroCouponBond>
  <transactionBuilder>ZeroCouponBondBuilder</transactionBuilder>
  <transactionID>CITI.0207</transactionID>
  <tradeDate>2008-07-02</tradeDate>
  <settlementDate>2008-07-05</settlementDate>
  <calendar>target</calendar>
  <faceAmount>1000000</faceAmount>
  <maturityDate>2032-07-05</maturityDate>
  <paymentConvention>modifiedfollowing</paymentConvention>
</ZeroCouponBond>

Needless to say, one should also add all required new conversion methods from their string presentation to QuantLib objects into previous module. The original C# implementation can be found in here. Thanks for reading my blog.
-Mike

Excel : implementing multithreading using C# and Excel-DNA

Multithreading possibilities for VBA has been chewed in this post. My conclusion back then was, that there is no multi-threading possibilities for VBA. One reader has then been commenting this as follows :

"Correct to say there is no multithreading in VBA however, you can very easily create your multithreading functionality in a c# dll and expose the interface via Interops for VBA to consume. Then all you need to do is to pass in the data into the function and walla, you have multithreading. Interops is a very powerful feature and VBA fully supports it."

For the sake of completeness, I will present one possible program to implement such multithreading scheme for Excel, by using Parallel class for the actual multi-threading part and Excel-DNA for interfacing the program back to Excel (using Excel only as Input-Output platform). Excel interfacing part used in this post, has already been completely chewed within this post. Note, that by using instructions given in here or here, one can relatively easy interface this multithreading program back to VBA, instead of Excel.

The program

This unsurprising program is simply processing a task (simulate N amount of normally distributed random numbers) shared for M processors. Amount of simulated numbers and number of processors used is decided by the client (Excel). Finally, the program will print the aggregated time consumed for this task back to Excel. Needless to say, the effect of multithreading can be demonstrated by changing the number of processors in Excel worksheet.

Copy-Paste the following program presented below into a new C# project. Also, follow carefully all instructions given in this post.

Thanks for reading this blog.
-Mike

// ExcelInterface.cs
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Threading.Tasks;
using System.Windows.Forms;
using ExcelDna.Integration;

namespace ExcelInterface {
    // type definition, delegate for exporting data from processor to queue
    using Processor = Action<int, DataMessenger>;
    public delegate void DataMessenger(List<double> result);

    public static class ExcelInterface {
        // method available in excel
        public static void execute() {
            try {
                // create Excel application object
                dynamic Excel = ExcelDnaUtil.Application;
                //
                // get parameters from worksheet named ranges
                int n = (int)Excel.Range["_n"].Value2;
                int nProcessors = (int)Excel.Range["_nProcessors"].Value2;
                int nPerProcessor = n / nProcessors;
                // create data thread-safe queue and random processors
                RandomQueue messageQueue = new RandomQueue();
                List<Processor> processors = new List<Processor>();
                for (int i = 0; i < nProcessors; ++i) {
                    processors.Add(Factory.CreateProcessor());
                }
                // host timer, execute all processors in parallel
                Stopwatch timer = new Stopwatch();
                timer.Start();
                Parallel.For(0, nProcessors, i => {
                    processors[i].Invoke(nPerProcessor, messageQueue.Enqueue); 
                });
                timer.Stop();
                //
                // print processing time to excel
                double timeElapsed = timer.Elapsed.TotalSeconds;
                Excel.Range["_timer"] = timeElapsed;
            }
            catch (Exception e) {
                MessageBox.Show(e.Message);
            }
        }
    }
}

// RandomQueue.cs
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;

namespace ExcelInterface {
    // wrapper class for ConcurrentQueue data structure
    public class RandomQueue {
        private ConcurrentQueue<List<double>> randomQueue;
        public RandomQueue() {
            randomQueue = new ConcurrentQueue<List<double>>();
        }
        // add
        public void Enqueue(List<double> data) {
            randomQueue.Enqueue(data);
        }
        // remove
        public List<double> Dequeue() {
            List<double> data = null;
            bool hasValue = false;
            while (!hasValue) hasValue = randomQueue.TryDequeue(out data);
            return data;
        }
    }
}

// Factory.cs
using System;
using System.Collections.Generic;

namespace ExcelInterface {
    // type definition
    using Processor = Action<int, DataMessenger>;
    
    public static class Factory {
        // factory method for creating processor for random generator
        public static Processor CreateProcessor() {
            //
            // create standard normal variable generator
            // using action delegate and lambda expression
            Processor process =
            (int n, DataMessenger dataMessenger) => {
                Random generator = new Random(Guid.NewGuid().GetHashCode());
                List<double> normalRandomList = new List<double>();
                // loop through n
                for (int i = 0; i < n; ++i) {
                    double uniformRandomSum = 0.0;
                    // create standard normal random variable using CLT
                    for (int j = 0; j < 12; ++j) {
                        double uniformRandom = 0.0;
                        while (true) {
                            uniformRandom = generator.NextDouble();
                            if (uniformRandom > 0.0) break;
                        }
                        uniformRandomSum += uniformRandom;
                    }
                    normalRandomList.Add(uniformRandomSum - 6.0);
                }
                // send list of normal random variables to queue
                dataMessenger(normalRandomList);
            };
            return process;
        }
    }
}

C++11 : Multi-Threaded PathGenerator using PPL

 

FINAL DESTINATION

The circle has been closed. This post is kind of an aggregation, based on the last four posts published on generating random numbers. Initially, I started just with a simple template class for distributional random generator, then continued with a path generator using any one-factor stochastic process and finally, ended up with a multi-threaded distributional random generation scheme using Parallel algorithms. This final post (hopefully) is opening up my larger goal : to be able to generate asset price paths for any one-factor process, using multi-threading scheme.

GROUNDHOG DAY

Again, I have tested the both sequential (for_each) and parallel (parallel_for_each) schemes by using four generators, 10000 paths and 250 time steps for a single run. After this, I repeated this run for 250 times. Conclusion :

• The average running time for this sample was 17116 milliseconds (sequential) and 8209 milliseconds (parallel). So, parallel scheme will be completed about two times faster. 
• The actual CPU usage profiles during the simulation processes are behaving exactly as reported in this post. 
• I also analyzed processed asset price paths for parallel scheme, just to be absolutely sure there are no path duplicates (random number generation would not be independent). Based on my analysis made in Excel, all processed asset price paths are different and there are no duplicates. 

Presented scheme for path generator is again fulfilling my two initial requirements : faster creation of asset price paths following any one-factor process and independency of random generators.

RandomGenerator.h

The basic functionality of this template class has not been changed, except for construction part : second constructor is allowing a client to give any probability distribution for uniform generator from outside of this class. Even there is actually no need for having this kind of optionality in real-life (most of the stuff in Monte Carlo is randomized by using standard normal distribution), I decided to implement this optionality for the sake of completeness.

#pragma once
#include <algorithm>
#include <functional>
#include <vector>
#include <random>
#include <memory>
//
namespace MikeJuniperhillRandomGeneratorTemplate
{
 template <typename Generator = std::mt19937, typename Distribution = std::normal_distribution<double>>
 /// <summary>
 /// Template class for creating random number paths using mt19937 as default uniform 
 /// random generator and Standard Normal as default probability distribution.
 /// </summary> 
 class RandomGenerator
 {
 public:
  /// <summary>
  /// Constructor with explicit seed value
  /// </summary>
  RandomGenerator(unsigned long seed)
  {
   // construct function for processing distributional random number
   randomGenerator = [this](double x)-> double
   {
    x = distribution(uniformGenerator);
    return x;
   };
   // seed generator once
   uniformGenerator.seed(seed);
  }
  /// <summary> 
  /// Constructor for explicit seed value and client-given probability distribution.
  /// </summary>  
  RandomGenerator(unsigned long seed, const Distribution& distribution)
   // constructor delegation
   : RandomGenerator(seed)
  {
   // assign client-given probability distribution
   this->distribution = distribution;
  }
  /// <summary>
  /// Fill a given vector reference with distributional random numbers
  /// </summary> 
  void operator()(std::vector<double>& v) const
  {
   std::transform(v.begin(), v.end(), v.begin(), randomGenerator);
  }
 private:
  std::function<double(double)> randomGenerator;
  Generator uniformGenerator;
  Distribution distribution;
 };
}
//

OneFactorProcess.h

I decided to tag drift and diffusion functions with const declaration, since these functions should not modify the internal state of class data members.

#pragma once
//
namespace MikeJuniperhillOneFactorProcessLibrary
{
 /// <summary>
 /// Abstract base class for all one-factor processes for customizing 
 /// drift and diffusion functions for different types of processes.
 /// </summary>
 class OneFactorProcess
 {
 public:
  virtual double drift(double x, double t) const = 0;
  virtual double diffusion(double x, double t) const = 0;
 };
 //
 /// <summary>
 /// Implementation for Vasicek short-rate model.
 /// </summary>
 class Vasicek : public OneFactorProcess
 {
 public:
  Vasicek(double meanReversion, double longTermRate, double rateVolatility)
   : meanReversion(meanReversion), longTermRate(longTermRate), rateVolatility(rateVolatility) { }
  //
  double drift(double x, double t) const override { return meanReversion * (longTermRate - x); }
  double diffusion(double x, double t) const override { return rateVolatility; }
 private:
  double meanReversion;
  double longTermRate;
  double rateVolatility;
 };
}
//

PathGenerator.h

As in the case with RandomGenerator, the basic functionality of this template class has not been changed either, except for construction part : second constructor is allowing a client to give any probability distribution to be delivered for distributional random generator.

#pragma once
//
#include "RandomGenerator.h"
#include "OneFactorProcess.h"
namespace MJRandom = MikeJuniperhillRandomGeneratorTemplate;
namespace MJProcess = MikeJuniperhillOneFactorProcessLibrary;
//
namespace MikeJuniperhillPathGenerator
{
 template <typename Generator = std::mt19937, typename Distribution = std::normal_distribution<double>>
 class PathGenerator
 {
 public:
  /// <summary>
  /// Constructor for PathGenerator template class.
  /// </summary>
  PathGenerator(double spot, double maturity, unsigned long seed,
   const std::shared_ptr<MJProcess::OneFactorProcess>& process)
   : spot(spot), maturity(maturity), process(process)
  {
   // create random generator
   generator = std::unique_ptr<MJRandom::RandomGenerator<Generator, Distribution>>
    (new MJRandom::RandomGenerator<Generator, Distribution>(seed));
  }
  /// <summary>
  /// Constructor for PathGenerator template class, with a client-given probability distribution
  /// </summary>
  PathGenerator(double spot, double maturity, unsigned long seed,
   const std::shared_ptr<MJProcess::OneFactorProcess>& process, const Distribution& distribution)
   : spot(spot), maturity(maturity), process(process)
  {
   // create random generator with client-given probability distribution
   generator = std::unique_ptr<MJRandom::RandomGenerator<Generator, Distribution>>
    (new MJRandom::RandomGenerator<Generator, Distribution>(seed, distribution));
  }
  /// <summary> 
  /// Fill a given vector reference with asset prices, following a given stochastic process.
  /// </summary>  
  void operator()(std::vector<double>& v) const
  {
   // transform initialized vector into a path containing random numbers
   (*generator)(v);
   //
   double dt = maturity / (v.size() - 1);
   double dw = 0.0;
   double s = spot;
   double t = 0.0;
   v[0] = s; // 1st path element will always be the current spot price
   //
   // transform distributional random number vector into a path containing 
   // asset prices from a given stochastic one-factor process
   for (auto it = v.begin() + 1; it != v.end(); ++it)
   {
    t += dt;
    dw = (*it) * std::sqrt(dt);
    (*it) = s + (*process).drift(s, t) * dt + (*process).diffusion(s, t) * dw;
    s = (*it);
   }
  }
 private:
  double spot;
  double maturity;
  std::shared_ptr<MJProcess::OneFactorProcess> process;
  std::unique_ptr<MJRandom::RandomGenerator<Generator, Distribution>> generator;
 };
}
//

Tester.cpp

Tester program is closely tracking the program presented in previous post. For the sake of additional clarity, I have used new type definitions in order to improve code readability and get rid of some lengthy variable names. The program is again using simple factory method for creating PathGenerator (function wrapped in shared pointer). In this program, OneFactorProcess implementation is created and delivered for factory method for processing. Finally, there is a method for printing processed paths to console for testing purposes.

#include <iostream>
#include <chrono>
#include <ppl.h>
#include <concurrent_vector.h>
#include "PathGenerator.h"
namespace MJGenerator = MikeJuniperhillPathGenerator;
//
// type definitions
using Path = std::vector<double>;
using Paths = concurrency::concurrent_vector<Path>;
using Process = std::shared_ptr<MJProcess::OneFactorProcess>;
using Processor = std::function<void(void)>;
using PathGenerator = std::shared_ptr<Processor>;
//
// thread-safe container for storing asset price paths, processed by path generators
Paths paths;
//
// printer for generated asset price paths
void Printer()
{
 std::for_each(paths.begin(), paths.end(),
  [](Path path) -> void
 {
  std::for_each(path.begin(), path.end(),
   [](double s) -> void
  {
   std::cout << s << ",";
  });
  std::cout << std::endl;
 });
}
//
// factory method :
// return path-generating function as function wrapper
// input arguments are common for all types of generators
PathGenerator Factory(double spot, double maturity, int nPaths, 
 int nSteps, unsigned long seed, const Process& process, Paths& paths)
{
 // create function for processing one-factor paths
 auto generator = [=, &process, &paths]() -> void
 {
  MJGenerator::PathGenerator<> oneFactorProcess(spot, maturity, seed, process);
  Path path(nSteps);
  for (auto i = 0; i != nPaths; ++i)
  {
   oneFactorProcess(path);
   paths.push_back(path);
  }
 };
 // return generator function as function wrapper
 return PathGenerator(new Processor(generator));
}
//
int main()
{
 // create vasicek process
 double longTermRate = 0.05;
 double meanReversion = 0.2;
 double rateVolatility = 0.0075; 
 Process vasicek = Process(new MJProcess::Vasicek(meanReversion, longTermRate, rateVolatility));
 //
 // define parameters and seed values for path generators
 int nGenerators = 4;
 int nPaths = 100;
 int nSteps = (250 + 1);
 std::vector<unsigned long> seed = { 10322854, 65947, 387528, 772399573 };
 //
 // use factory method for creating path generators
 double spot = 0.0095;
 double maturity = 3.0;
 std::vector<PathGenerator> generators;
 for (auto i = 0; i < nGenerators; i++) generators.push_back(
  Factory(spot, maturity, nPaths, nSteps, seed[i], vasicek, paths));
 //
 // parallel processing
 auto start = std::chrono::steady_clock::now();
 concurrency::parallel_for_each(generators.begin(), generators.end(),
  [](PathGenerator pg) -> void { (*pg)(); });
 auto end = std::chrono::steady_clock::now();
 auto timeElapsed = std::chrono::duration_cast<std::chrono::milliseconds>(end - start);
 std::cout << timeElapsed.count() << std::endl;
 //
 // print paths
 Printer();
 return 0;
}
//

Finally, thanks again for reading this blog.
-Mike

C++11 : Multi-Threaded Random Generator using PPL

After some woodshedding with C++11 random and publishing distributional random generator in my blog a couple of weeks ago, the next question was, how to make this thing to happen a bit faster? Since I already have been snooping PPL (Parallel Patterns Library), I realized that Parallel Algorithms in that library would be an excellent candidate for this task. There is a great and comprehensive internet book available on this topic. After getting familiar with some of the issues presented in these sources, one should be able to parallelize algorithms succesfully.

There are some very specific issues, related to random generator seeding. As I started to parallelize my random number generation (creating several generators and processing these using parallel algorithm), I realized that my random generators were not working as expected. After some further woodshedding I came to conclusion, that (in a multi-threaded setting) each generator has to be seeded independently. Finally, concerning my current generator implementation, I made a decision that a client has to provide explicit seed values for all generators. Based on my hands-on experience with a system used for counterparty credit risk simulations (widely ranked as state-of-the-art numerical software), I dare to say that this approach is well justified. Now, the question of what kind of tools will be used for creating the actual seed values, has been left to a client judgement. The rabbit hole of these profound issues related to random generators have been chewed in this blog. Do not enter, if you are not absolutely ready to take the red pill.

ROUGH DIAGNOSTICS

I tested the both sequential (for_each) and parallel (parallel_for_each) schemes by using four generators, 10000 paths and 250 time steps for a single run. After this, I repeated this run for 250 times. The average running time for this sample was 5187 milliseconds (sequential) and 2317 milliseconds (parallel). So, parallel scheme will be completed about two times faster. The actual CPU usages during simulation process for the both schemes are shown in the screenshot below. When using parallel algorithm, computer is able to utilize all four cores in my computer.

I also tested all processed random paths for the both schemes, just to be absolutely sure there are no path duplicates (random number generation would not be independent). Based on my analysis made in Excel, all processed paths for the both schemes are different and there are no duplicates. Presented scheme is hereby fulfilling my two requirements : faster creation of random paths and independency of random generators.

RandomGenerator.h

Current implementation for distributional random path generator is presented below. Note, that the actual seeder creation takes place outside of this class.

#pragma once
#include <algorithm>
#include <functional>
#include <vector>
#include <random>
//
namespace MikeJuniperhillRandomGeneratorTemplate
{
 template <typename Generator = std::mt19937, typename Distribution = std::normal_distribution<double>>
 /// <summary>
 /// Template class for creating random number paths using
 /// Mersenne Twister as default uniform random generator and
 /// Standard Normal (0.0, 1.0) as default probability distribution.
 /// </summary> 
 class RandomGenerator
 {
 public:
  /// <summary>
  /// set explicit seed value
  /// </summary>
  RandomGenerator(unsigned long seed)
  {
   // construct lambda method for processing standard normal random number
   randomGenerator = [this](double x)-> double
   {
    x = distribution(uniformGenerator);
    return x;
   };
   // seed generator
   uniformGenerator.seed(seed);
  }
  /// <summary>
  /// fill vector reference with distributional random numbers
  /// </summary> 
  void operator()(std::vector<double>& v)
  {
   std::transform(v.begin(), v.end(), v.begin(), randomGenerator);
  }
 private:
  std::function<double(double)> randomGenerator;
  Generator uniformGenerator;
  Distribution distribution;
 };
}
//

tester.cpp

First, all simulation parameters and explicit seed values for all generators are defined. Simple factory method is then used to create the actual generator function as lambda function. Next, factory method is returning generator function wrapped inside a shared pointer. Main program is then storing each created generator function into vector container and finally processing all generators (sequentially or parallel). Each generator is storing generated paths into thread-safe nested vector (std::vector inside concurrency::concurrent_vector).

#include <iostream>
#include <memory>
#include <chrono>
#include <ppl.h>
#include <concurrent_vector.h>
#include "RandomGenerator.h"
namespace MJ = MikeJuniperhillRandomGeneratorTemplate;
//
//
// thread-safe nested vector for storing random paths processed by generators
concurrency::concurrent_vector<std::vector<double>> paths;
//
// prototype for generator function
using Generator = std::function<void(void)> ;
//
// factory method :
// return generator function as function wrapper
std::shared_ptr<Generator> Factory(int nPaths, int nSteps, unsigned long seed, 
 concurrency::concurrent_vector<std::vector<double>>& paths)
{
 // create generator function
 auto generator = [=, &paths]()->void
 {
  // step 1 : create distributional random generator
  MJ::RandomGenerator<> randomGenerator(seed);
  // step 2 : create container for a random path
  std::vector<double> path(nSteps);
  // step 3 : create path using distributional generator
  // and import created path into thread-safe container
  for (auto i = 0; i != nPaths; ++i)
  {
   randomGenerator(path);
   paths.push_back(path);
  }
 };
 // return generator function as function wrapper
 return std::shared_ptr<Generator>(new Generator(generator));
}
//
int main()
{
 // define parameters and seed values for generator functions
 int nGenerators = 4;
 int nPaths = 10000;
 int nSteps = 250;
 std::vector<unsigned long> seed = { 10322854, 65947, 387528, 772399573 };
 //
 // use factory method for creating generator function
 std::vector<std::shared_ptr<Generator>> generators;
 for (auto i = 0; i < nGenerators; i++) generators.push_back(Factory(nPaths, nSteps, seed[i], paths));
 //
 // parallel processing
 auto start = std::chrono::steady_clock::now();
 concurrency::parallel_for_each(generators.begin(), generators.end(),
  [](std::shared_ptr<Generator> ppg) -> void { (*ppg)(); });
 //
 //// uncomment for sequential processing
 //std::for_each(generators.begin(), generators.end(),
 // [](std::shared_ptr<Generator> ppg) -> void { (*ppg)(); });
 //
 auto end = std::chrono::steady_clock::now();
 auto timeElapsed = (end - start);
 std::cout << std::chrono::duration_cast<std::chrono::milliseconds>(timeElapsed).count() << std::endl;
 return 0;
}
//

Finally, thanks for reading my blog. I would like to wish Happy New Year for everybody.
-Mike

Pricing Bloomberg Swap Manager Transactions with C#

From time to time, I need to have reliable second opinion or benchmark valuation for a given book of swap transactions. Bloomberg Swap Manager (SWPM) is just a great tool for pricing many different kinds of swaps and other types of transactions. In this small project, we are constructing a program, which can be used to request valuation fields for several portfolios (one portfolio is a list of Swap Manager transactions ID's stored in source file). Program is using Bloomberg C# API for requesting valuation fields from Bloomberg.

SWAP MANAGER AND FIELD SEARCH

Let us assume, that the following cross-currency basis swap transaction has been created and stored in Bloomberg Swap Manager. Note, that when the transaction is stored, Swap Manager is automatically creating identification code for this transaction (SL5Q3637 - on the upper right corner of the screen).

When using Bloomberg Field Search (FLDS) for this transaction (SL5Q3637 Corp), I can request all the fields (static, calculated) what are available in any of the Swap Manager screens. Just as one example, we see that in the transaction screen above, dirty PV for this swap is 12 157 660 EUR (for historical valuation date 5.6.2015).

Field search is showing the value of  11 889 635 EUR (SW_MARKET_VAL). However, since the transaction valuation date is historical, we need to override settlement and curve date. Clicking SW_MARKET_VAL (dirty PV) field in the Field Search screen will show all the parameters, what are available for overriding for this particular field.

By overriding SETTLE_DT and SW_CURVE_DT,  PV (SW_MARKET_VAL) is 12 157 660 EUR, which is exactly the same as in our initial transaction screen above. Note, that if you are valuing any transaction per historical date, the both fields (settlement data, curve date) have to have the same date value.

Now we come to the beef of this posting : since we can use Bloomberg Field Search for finding a value for any fields what are available in any of the Swap Manager screens, we can also request values for those fields by using Bloomberg API. The easiest way is just to use Excel and Bloomberg-addin function BDP (security = SL5Q3637 Corp, field = SW_MARKET_VAL). Another way is to request those fields programmatically, by using Bloomberg programming API. In this posting, we are using the latter approach.

PROGRAM INPUT AND OUTPUT

A list of swap transactions (which have already been created and stored in Swap Manager) for a given portfolio (tradingSwaps in this example) is defined in the transaction source file (csv).

The program is producing the following output for a given portfolio of swap transactions (using current configurations).

CONFIGURATION FILES

Application configurations file (App.Config in C# project) is only defining repository folder for all individual portfolio configurations. It should be noted, that we can have several different portfolio configurations at the same time in that repository folder.

Single portfolio configuration file (as shown in the picture below) defines the following information

• Portfolio name
• Source data path and file name
• Field separator for source file
• Bloomberg fields to be requested
• Possible Bloomberg override fields
• Possible Bloomberg override values
• Type of target output (file, console, excel, etc)
• Path and file name for output file (if not console)

By handling portfolio construction via individual configuration files, we are achieving a lot of flexibility into this program. For example, different portfolios can

• read source data (Bloomberg transaction ID's) from any given source data file.
• have completely different set of Bloomberg request fields and set of field overrides.
• be printed out into different target medium (console, csv, xml, web-page, Excel, etc).

In a nutshell, the program is first finding configurations repository folder based on the information given in Application configurations file (App.Config). After this, the program is constructing N portfolio configuration objects for all existing configuration files given in repository folder. Finally, program is using Bloomberg API for requesting data for each configuration and then printing out the results based on given configuration.

PROGRAM DESIGN

UML class diagram has been presented below.

GREEN
In the first stage, Configuration objects has to be created. In this example program, Client is sending creation request for static Configurations class. Static Configurations class is then using concrete XMLConfigurationBuilder (implementation of abstract ConfigurationBuilder) to create Configuration objects. Static Configuration class has one to many Configuration objects. Static Configurations class is available for all the other objects during the program execution.

THISTLE
In the second stage, PortfolioProcessor object has to be created. Client is sending creation request for one concrete FileProcessorBuilder (implementation of abstract PortfolioProcessorBuilder) to build one PortfolioProcessor object. On a general level, PortfolioProcessor is hosting several other objects: one to many Portfolio objects (aggregation), one BBCOMMWrapper object (composition) and one to many PortfolioVisitor objects (aggregation).

YELLOW
PortfolioProcessor has one BBCOMMWrapper object (composition) for requesting data from Bloomberg by using BBCOMM Server API. For any portfolio being processed, all data fields (and optional field overrides), which are going to be requested from Bloomberg API, have been defined inside static Configurations class.

TURQUOISE
There is a hierarchy of objects. PortfolioProcessor object has one to many Portfolio objects (aggregation). One Portfolio object has one to many Contract objects (aggregation). One Contract object has one to many Field objects (aggregation). Essentially, when PortfolioProcessor is executing its Process method, it will request results from BBCOMMWrapper object for all contracts and data fields, for all Portfolio objects which are hosted inside PortfolioProcessor object. After receiving results from BBCOMMWrapper object, it will then export all results into corresponding Field objects for each Contract and Portfolio.

WHEAT
When PortfolioProcessor object is being built, concrete FileProcessorBuilder is also using PortfolioVisitorFactory object for creating one or more concrete Visitor objects. References of those Visitor objects are stored inside PortfolioProcessor (aggregation). Essentially, PortfolioProcessor object method Print will print all results (values from stored in Field objects inside Contract objects for each Portfolio object) into selected target medium. For this task, PortfolioProcessor object is using one or more Visitor objects (specific Visitor type for each Portfolio).

THE PROGRAM

It should be noted, that each class (or class hierarchy) should be copy-pasted into a separate cs-file. References to Bloomberg API and System.Configuration should be made in order to compile this program.

// file : Program.cs
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Configuration;
using System.IO;
using System.Xml;

namespace PortfolioValuation
{
    // CLIENT
    class Program
    {
        static void Main(string[] args)
        {
            try
            {
                // build static configurations from pre-defined configuration files for each portfolio
                Configurations.Build(new XMLConfigurationBuilder());
                //
                // build swap manager object from a given file information
                PortfolioProcessorBuilder swapManagerBuilder = new FileProcessorBuilder();
                PortfolioProcessor swapManager = swapManagerBuilder.Build();
                //
                // process and print portfolio
                swapManager.Process();
                swapManager.Print();
            }
            catch (Exception e)
            {
                Console.WriteLine(e.Message);
            }
            Console.WriteLine("Program has been executed");
            Console.ReadLine();
        }
    }
}
//
//
// file : PortfolioProcessorBuilder.cs
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Configuration;

namespace PortfolioValuation
{
    // builder for swap manager object - class hierarchy
    public abstract class PortfolioProcessorBuilder
    {
        public abstract PortfolioProcessor Build();
    }
    // build data for swap manager object using source files
    public class FileProcessorBuilder : PortfolioProcessorBuilder
    {
        public override PortfolioProcessor Build()
        {
            // build portfolio and visitor objects based on configurations
            List<Portfolio> portfolios = new List<Portfolio>();
            List<PortfolioVisitor> visitors = new List<PortfolioVisitor>();
            //
            // loop through configurations
            for (int i = 0; i < Configurations.N; i++)
            {
                // get one configuration, create portfolio object and
                // read source data (isin codes) for this portfolio
                Configuration table = Configurations.Get(i);
                string ID = table.ID;
                Portfolio portfolio = new Portfolio(ID);
                List<string> sourceData = new List<string>();
                TextFileHandler.Read(table.Get("sourcedatafilepathname"), sourceData);
                //
                // create empty contracts into portfolio
                char fieldSeparator = Convert.ToChar(Configurations.Get(ID, "fieldseparator"));
                sourceData.ForEach(it =>
                {
                    if (it.Split(Convert.ToChar(fieldSeparator))[0] == ID)
                        portfolio.AddContract(new Contract(it.Split(fieldSeparator)[1]));
                });
                //
                // decision : portfolio must have at least one contract
                if (portfolio.Count() == 0) throw new Exception("builder : empty portfolio error");
                //
                // request visitor for swap manager object, created by factory method
                PortfolioOutputVisitorFactory factory = new PortfolioOutputVisitorFactory();
                string visitorType = table.Get("visitortype");
                PortfolioVisitor visitor = factory.CreatePortfolioOutputVisitor(visitorType);
                portfolios.Add(portfolio);
                visitors.Add(visitor);
            }
            // return constructed swap manager objects for a client
            return new PortfolioProcessor(portfolios, visitors);
        }
    }
}
//
//
// file : PortfolioProcessor.cs
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace PortfolioValuation
{
    public class PortfolioProcessor
    {
        private BBCOMMDataRequest bloombergAPI;
        private List<PortfolioVisitor> visitors;
        private List<Portfolio> portfolios;
        private List<string> securities;
        private List<string> bloombergFields;
        private List<string> bloombergOverrideFields;
        private List<string> bloombergOverrideValues;
        //
        public PortfolioProcessor(List<Portfolio> portfolios, List<PortfolioVisitor> visitors)
        {
            this.portfolios = portfolios;
            this.visitors = visitors;
        }
        public void Process()
        {
            // process all portfolios
            foreach (Portfolio portfolio in portfolios)
            {
                // create information for Bloomberg API
                string ID = portfolio.ID;
                char fieldSeparator = Convert.ToChar(Configurations.Get(ID, "fieldseparator"));
                securities = portfolio.Select(it => it.Isin).ToList();
                //
                bloombergFields = Configurations.Get(ID, "bloombergfields").Split(fieldSeparator).ToList<string>();
                bloombergOverrideFields = Configurations.Get(ID, "bloombergoverridefields").Split(fieldSeparator).ToList<string>();
                bloombergOverrideValues = Configurations.Get(ID, "bloombergoverridevalues").Split(fieldSeparator).ToList<string>();
                //
                // decision : override collections always has to be provided
                if ((bloombergOverrideFields == null) || (bloombergOverrideValues == null))
                    throw new Exception("swap manager : null override collection error");
                //
                // create reference data request for Bloomberg API
                if ((bloombergOverrideFields.Count != 0) && (bloombergOverrideValues.Count != 0))
                {
                    // data request with overrides
                    bloombergAPI = new ReferenceDataRequest(securities, bloombergFields, bloombergOverrideFields, bloombergOverrideValues);
                }
                else
                {
                    // data request without overrides
                    bloombergAPI = new ReferenceDataRequest(securities, bloombergFields);
                }
                // receive requested data
                dynamic[, ,] result = bloombergAPI.ProcessData();
                //
                // process bloomberg results into portfolio contracts and fields
                int nFields = bloombergFields.Count;
                int nSecurities = securities.Count;
                for (int i = 0; i < nSecurities; i++)
                {
                    for (int j = 0; j < nFields; j++)
                    {
                        Field field = new Field(bloombergFields[j], result[i, 0, j]);
                        portfolio.UpdateContract(securities[i], field);
                    }
                }
            }
        }
        public void Print()
        {
            // delegate accept requests for portfolio objects
            for (int i = 0; i < portfolios.Count; i++)
            {
                portfolios[i].AcceptVisitor(visitors[i]);
            }
        }
    }
}
//
//
// file : TextFileHandler.cs
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.IO;

namespace PortfolioValuation
{
    public static class TextFileHandler
    {
        public static void Read(string filePathName, List<string> output)
        {
            // read file content into list
            StreamReader reader = new StreamReader(filePathName);
            while (!reader.EndOfStream)
            {
                output.Add(reader.ReadLine());
            }
            reader.Close();
        }
        public static void Write(string filePathName, List<string> input, bool append)
        {
            // write text stream list to file
            StreamWriter writer = new StreamWriter(filePathName, append);
            input.ForEach(it => writer.WriteLine(it));
            writer.Close();
        }
    }
}
//
//
// file : PortfolioVisitor.cs
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Configuration;

namespace PortfolioValuation
{
    // output visitor - class hierarchy
    public abstract class PortfolioVisitor
    {
        public abstract void Visit(Portfolio portfolio);
    }
    public class ConsolePortfolioVisitor : PortfolioVisitor
    {
        public override void Visit(Portfolio portfolio)
        {
            //Configuration configuration
            foreach (Contract contract in portfolio)
            {
                string fieldSeparator = Configurations.Get(portfolio.ID, "fieldseparator");
                StringBuilder sb = new StringBuilder();
                foreach (Field f in contract)
                {
                    sb.Append(f.Value);
                    sb.Append(fieldSeparator);
                }
                // print the content of string builder into console
                Console.WriteLine(sb.ToString());
            }
        }
    }
    public class FilePortfolioVisitor : PortfolioVisitor
    {
        public override void Visit(Portfolio portfolio)
        {
            // extract filePathName for a given portfolio from configuration table
            string fieldSeparator = Configurations.Get(portfolio.ID, "fieldseparator");
            List<string> resultDataFilePathNames = Configurations.Get(portfolio.ID, "resultdatafilepathname").Split(Convert.ToChar(fieldSeparator)).ToList();
            List<string> portfolioIDs = Configurations.Get(portfolio.ID, "portfolioid").Split(Convert.ToChar(fieldSeparator)).ToList();
            string resultDataFilePathName = null;
            for (int i = 0; i < portfolioIDs.Count; i++)
            {
                if(portfolioIDs[i] == portfolio.ID)
                {
                    resultDataFilePathName = resultDataFilePathNames[i];
                    break;
                }
            }
            List<string> fileOutputList = new List<string>();
            StringBuilder sb = new StringBuilder();
            //
            foreach (Contract contract in portfolio)
            {
                // add all fields into string builder object
                foreach (Field f in contract)
                {
                    sb.Append(f.Value);
                    sb.Append(fieldSeparator);
                }
                // add constructed string into output list and clear string builder
                fileOutputList.Add(sb.ToString());
                sb.Clear();
            }
            // print the content of output list into file
            TextFileHandler.Write(resultDataFilePathName, fileOutputList, false);
        }
    }
}
//
//
// file : PortfolioConfiguration.cs
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace PortfolioValuation
{
    public class PortfolioConfiguration
    {
        // private member data
        public readonly string ID;
        private Dictionary<string, string> configurationTable = new Dictionary<string, string>();
        //
        public PortfolioConfiguration(string ID)
        {
            this.ID = ID;
        }
        // add key-value pair
        public void Add(string key, string value)
        {
            // decision : configuration table cannot contain two identical keys
            if (configurationTable.ContainsKey(key))
                throw new Exception("configuration table : duplicate key error");
            configurationTable.Add(key, value);
        }
        // get value for a given key
        public string Get(string key)
        {
            // decision : value for a given key cannot be returnde if it does not exist
            if (!configurationTable.ContainsKey(key))
                throw new Exception("configuration table : invalid key error");
            return configurationTable[key];
        }
        // clear all key-value pairs
        public void Clear()
        {
            // clear all configurations
            configurationTable.Clear();
        }
    }
}
//
//
// file : Portfolio.cs
using System;
using System.Collections.Generic;
using System.Collections;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace PortfolioValuation
{
    public class Portfolio : IEnumerable<Contract>
    {
        // private member data
        public readonly string ID;
        private List<Contract> contracts;
        //
        // ctor
        public Portfolio(string ID)
        {
            this.ID = ID;
            contracts = new List<Contract>();
        }
        // add one contract into list of contracts
        public void AddContract(Contract contract)
        {
            // decision : portfolio cannot contain two identical contracts
            if(contracts.Exists(it => it.Isin == contract.Isin))
                throw new Exception("portfolio : duplicate contract error");
            contracts.Add(contract);
        }
        // update existing contract in list of contracts
        public void UpdateContract(string isin, Field field)
        {
            // contract cannot be updated if it does not exist
            if (!contracts.Exists(it => it.Isin == isin))
                throw new Exception("portfolio : update contract error");
            //
            for (int i = 0; i < contracts.Count; i++)
            {
                if (contracts[i].Isin == isin)
                {
                    contracts[i].AddField(field);
                    break;
                }
            }
        }
        // implementation for generic IEnumerable
        public IEnumerator<Contract> GetEnumerator()
        {
            foreach (Contract contract in contracts)
            {
                yield return contract;
            }
        }
        // implementation for non-generic IEnumerable
        IEnumerator IEnumerable.GetEnumerator()
        {
            return GetEnumerator();
        }
        // implementation method for visitor pattern
        public void AcceptVisitor(PortfolioVisitor visitor)
        {
            // send this-object (portfolio) for visitor implementation
            visitor.Visit(this);
        }
    }
}
//
//
// file : PortfolioOutputVisitorFactory.cs
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace PortfolioValuation
{
    public class PortfolioOutputVisitorFactory
    {
        // create output visitor
        public PortfolioVisitor CreatePortfolioOutputVisitor(string visitorType)
        {
            // HARD-CODED selection of output visitors
            // add new visitor type into switch branch and write 
            // implementation into visitor class hierarchy
            PortfolioVisitor visitor = null;
            switch (visitorType)
            {
                case "console":
                    visitor = new ConsolePortfolioVisitor();
                    break;
                case "file":
                    visitor = new FilePortfolioVisitor();
                    break;
                default:
                    throw new Exception("factory : undefined visitor error");
            }
            return visitor;
        }
    }
}
//
//
// file : Field.cs
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace PortfolioValuation
{
    public class Field
    {
        // private data and read-only accessors
        private KeyValuePair<string, dynamic> kvp;
        public string Key { get { return kvp.Key; } }
        public dynamic Value { get { return kvp.Value; } }
        //
        // ctor
        public Field(string key, dynamic value)
        {
            kvp = new KeyValuePair<string, dynamic>(key, value);
        }
    }
}
//
//
// file : Contract.cs
using System;
using System.Collections.Generic;
using System.Collections;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace PortfolioValuation
{
    public class Contract : IEnumerable<Field>
    {
        // private data and read-only accessor
        private string isin;
        private List<Field> fields;
        public string Isin { get { return isin; } }
        //
        // ctor
        public Contract(string isin)
        {
            this.isin = isin;
            fields = new List<Field>();
        }
        // add one field into list of fields
        public void AddField(Field field)
        {
            // decision : contract cannot contain two identical fields
            if (fields.Exists(it => it.Key == field.Key))
                throw new Exception("contract : duplicate field error");
            fields.Add(field);
        }
        // implementation for generic IEnumerable
        public IEnumerator<Field> GetEnumerator()
        {
            foreach (Field field in fields)
            {
                yield return field;
            }
        }
        // implementation for non-generic IEnumerable
        IEnumerator IEnumerable.GetEnumerator()
        {
            return GetEnumerator();
        }
    }
}
//
//
// file : Configurations.cs
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace PortfolioValuation
{
    // static wrapper class for all configurations
    public static class Configurations
    {
        // static members and accessors
        private static int n;
        public static int N { get { return Configurations.n; } }
        private static List<Configuration> tables = null;
        //
        // ctor
        public static void Build(ConfigurationBuilder builder)
        {
            tables = builder.Build();
            n = tables.Count;
        }
        public static string Get(string ID, string key)
        {
            string value = null;
            foreach (Configuration table in tables)
            {
                if (table.ID == ID)
                {
                    value = table.Get(key);
                    break;
                }
            }
            return value;
        }
        public static Configuration Get(int index)
        {
            return tables[index];
        }
    }
}
//
//
// file : ConfigurationBuilder.cs
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Xml;
using System.Xml.Linq;
using System.IO;
using System.Configuration;

namespace PortfolioValuation
{
    // class hierarchy for configuration builder
    public abstract class ConfigurationBuilder
    {
        public abstract List<Configuration> Build();
    }
    public class XMLConfigurationBuilder : ConfigurationBuilder
    {
        private string folderPath;
        private List<Configuration> configurations;
        public XMLConfigurationBuilder()
        {
            // read path to source folder containing configurations for all portfolios directly from system configuration file
            this.folderPath = ConfigurationManager.AppSettings["PortfolioConfigurationsFolder"].ToString();
            configurations = new List<Configuration>();
        }
        public override List<Configuration> Build()
        {
            foreach (string filePathName in Directory.GetFiles(folderPath))
            {
                XDocument database = XDocument.Load(filePathName);
                Dictionary<string, string> pairs = database.Descendants("configuration")
                    .Elements().ToDictionary(r => r.Attribute("key").Value.ToString(),
                    r => r.Attribute("value").Value.ToString());
                configurations.Add(new Configuration(pairs["portfolioid"], pairs));
            }
            return configurations;
        }
    }
}
//
//
// file : Configuration.cs
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace PortfolioValuation
{
    public class Configuration
    {
        public readonly string ID;
        private Dictionary<string, string> pairs;
        //
        // ctor
        public Configuration(string ID, Dictionary<string, string> pairs)
        {
            this.ID = ID;
            this.pairs = pairs;
        }
        // accessor method
        public string Get(string key)
        {
            // decision : value for a given key cannot be returned if it does not exist
            if (!pairs.ContainsKey(key))
                throw new Exception("configuration : invalid key error"); 
            return this.pairs[key];
        }
    }
}
//
//
// file : BBCOMMWrapper.cs
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using BBCOMM = Bloomberglp.Blpapi;

namespace PortfolioValuation
{
    // enumerators for historical data request settings
    public enum E_PRICING_OPTION { PRICING_OPTION_PRICE, PRICING_OPTION_YIELD };
    public enum E_PERIODICITY_ADJUSTMENT { ACTUAL, CALENDAR, FISCAL };
    public enum E_PERIODICITY_SELECTION { DAILY, WEEKLY, MONTHLY, QUARTERLY, SEMI_ANNUALLY, YEARLY };
    public enum E_NON_TRADING_DAY_FILL_OPTION { NON_TRADING_WEEKDAYS, ALL_CALENDAR_DAYS, ACTIVE_DAYS_ONLY };
    public enum E_NON_TRADING_DAY_FILL_METHOD { PREVIOUS_VALUE, NIL_VALUE };
    //
    // abstract base class for data request
    public abstract class BBCOMMDataRequest
    {
        // BBCOMM names
        protected readonly BBCOMM.Name SECURITY_DATA = new BBCOMM.Name("securityData");
        protected readonly BBCOMM.Name FIELD_DATA = new BBCOMM.Name("fieldData");
        protected readonly BBCOMM.Name FIELD_ID = new BBCOMM.Name("fieldId");
        protected readonly BBCOMM.Name VALUE = new BBCOMM.Name("value");
        protected readonly BBCOMM.Name OVERRIDES = new BBCOMM.Name("overrides");
        protected readonly BBCOMM.Name SECURITIES = new BBCOMM.Name("securities");
        protected readonly BBCOMM.Name FIELDS = new BBCOMM.Name("fields");
        protected readonly BBCOMM.Name SEQUENCE_NUMBER = new BBCOMM.Name("sequenceNumber");
        protected readonly BBCOMM.Name START_DATE = new BBCOMM.Name("startDate");
        protected readonly BBCOMM.Name END_DATE = new BBCOMM.Name("endDate");
        protected readonly BBCOMM.Name DATE = new BBCOMM.Name("date");
        protected readonly BBCOMM.Name PRICING_OPTION = new BBCOMM.Name("pricingOption");
        protected readonly BBCOMM.Name PERIODICITY_ADJUSTMENT = new BBCOMM.Name("periodicityAdjustment");
        protected readonly BBCOMM.Name PERIODICITY_SELECTION = new BBCOMM.Name("periodicitySelection");
        protected readonly BBCOMM.Name NON_TRADING_DAY_FILL_OPTION = new BBCOMM.Name("nonTradingDayFillOption");
        protected readonly BBCOMM.Name NON_TRADING_DAY_FILL_METHOD = new BBCOMM.Name("nonTradingDayFillMethod");
        protected readonly BBCOMM.Name OVERRIDE_CURRENCY = new BBCOMM.Name("currency");
        //
        // const strings, enumerators, etc.
        protected readonly string NOT_AVAILABLE = "#N/A";
        protected readonly string SESSION_EXCEPTION = "Session not started";
        protected readonly string SERVICE_EXCEPTION = "Service not opened";
        protected readonly string REQUEST_TYPE_REFERENCE = "ReferenceDataRequest";
        protected readonly string REQUEST_TYPE_HISTORICAL = "HistoricalDataRequest";
        protected readonly string REFERENCE_DATA_SERVICE = "//blp/refdata";
        protected readonly string BLOOMBERG_DATE_FORMAT = "yyyyMMdd";
        protected E_PRICING_OPTION pricingOption;
        protected E_PERIODICITY_ADJUSTMENT periodicityAdjustment;
        protected E_PERIODICITY_SELECTION periodicitySelection;
        protected E_NON_TRADING_DAY_FILL_OPTION nonTradingDayFillOption;
        protected E_NON_TRADING_DAY_FILL_METHOD nonTradingDayFillMethod;
        protected string requestType;
        protected string startDate;
        protected string endDate;
        protected string overrideCurrency;
        //
        // wrapped BBCOMM objects
        protected BBCOMM.Session session;
        protected BBCOMM.Service service;
        protected BBCOMM.Request request;
        //
        // input data structures
        protected List<string> securityNames = new List<string>();
        protected List<string> fieldNames = new List<string>();
        protected List<string> overrideFields = new List<string>();
        protected List<string> overrideValues = new List<string>();
        //
        // output result data structure
        protected dynamic[, ,] result;
        //
        public dynamic[, ,] ProcessData()
        {
            Open();
            CreateRequest();
            SendRequest();
            Close();
            return result;
        }
        private void Open()
        {
            // create and start bloomberg BBCOMM session
            BBCOMM.SessionOptions sessionOptions = new BBCOMM.SessionOptions();
            session = new BBCOMM.Session(sessionOptions);
            if (!session.Start()) throw new Exception(SESSION_EXCEPTION);
            //
            // get service from session object and create request by service object
            if (!session.OpenService(REFERENCE_DATA_SERVICE)) throw new Exception(SERVICE_EXCEPTION);
            service = session.GetService(REFERENCE_DATA_SERVICE);
            request = service.CreateRequest(requestType);
        }
        private void CreateRequest()
        {
            // append securities, fields
            foreach (string securityName in securityNames) request.Append(SECURITIES, securityName);
            foreach (string fieldName in fieldNames) request.Append(FIELDS, fieldName);
            //
            // conditionally, append overrides into request object
            if (overrideFields.Count > 0)
            {
                BBCOMM.Element requestOverrides = request.GetElement(OVERRIDES);
                for (int i = 0; i < overrideFields.Count; i++)
                {
                    BBCOMM.Element requestOverride = requestOverrides.AppendElement();
                    requestOverride.SetElement(FIELD_ID, overrideFields[i]);
                    requestOverride.SetElement(VALUE, overrideValues[i]);
                }
            }
            // set optional parameters for historical data request
            if (requestType == REQUEST_TYPE_HISTORICAL)
            {
                request.Set(START_DATE, startDate);
                request.Set(END_DATE, endDate);
                request.Set(PRICING_OPTION, pricingOption.ToString());
                request.Set(PERIODICITY_ADJUSTMENT, periodicityAdjustment.ToString());
                request.Set(PERIODICITY_SELECTION, periodicitySelection.ToString());
                request.Set(NON_TRADING_DAY_FILL_OPTION, nonTradingDayFillOption.ToString());
                request.Set(NON_TRADING_DAY_FILL_METHOD, nonTradingDayFillMethod.ToString());
                if (overrideCurrency != String.Empty) request.Set(OVERRIDE_CURRENCY, overrideCurrency);
            }
        }
        private void SendRequest()
        {
            // send constructed request to BBCOMM server
            long ID = Guid.NewGuid().GetHashCode();
            session.SendRequest(request, new BBCOMM.CorrelationID(ID));
            bool isProcessing = true;
            //
            while (isProcessing)
            {
                // receive data response from BBCOMM server, send 
                // response to be processed by sub-classed algorithm
                BBCOMM.Event response = session.NextEvent();
                switch (response.Type)
                {
                    case BBCOMM.Event.EventType.PARTIAL_RESPONSE:
                        ProcessDataResponse(ref response);
                        break;
                    case BBCOMM.Event.EventType.RESPONSE:
                        ProcessDataResponse(ref response);
                        isProcessing = false;
                        break;
                    default:
                        break;
                }
            }
        }
        private void Close()
        {
            // close BBCOMM session
            if (session != null) session.Stop();
        }
        //
        // sub-classes are providing specific algorithm implementations for 
        // processing and packing BBCOMM server response data into resulting data structure
        protected abstract void ProcessDataResponse(ref BBCOMM.Event response);
    }
    //
    // concrete class implementation for processing reference data request
    public class ReferenceDataRequest : BBCOMMDataRequest
    {
        public ReferenceDataRequest(List<string> bloombergSecurityNames,
            List<string> bloombergFieldNames)
        {
            // ctor : create reference data request without field overrides
            requestType = REQUEST_TYPE_REFERENCE;
            securityNames = bloombergSecurityNames;
            fieldNames = bloombergFieldNames;
            //
            // define result data structure dimensions for reference data request
            result = new dynamic[securityNames.Count, 1, fieldNames.Count];
        }
        public ReferenceDataRequest(List<string> bloombergSecurityNames,
            List<string> bloombergFieldNames, List<string> bloombergOverrideFields,
            List<string> bloombergOverrideValues)
        {
            // ctor : create reference data request with field overrides
            requestType = REQUEST_TYPE_REFERENCE;
            securityNames = bloombergSecurityNames;
            fieldNames = bloombergFieldNames;
            overrideFields = bloombergOverrideFields;
            overrideValues = bloombergOverrideValues;
            //
            // define result data structure dimensions for reference data request
            result = new dynamic[securityNames.Count, 1, fieldNames.Count];
        }
        protected override void ProcessDataResponse(ref BBCOMM.Event response)
        {
            // receive response, which contains N securities and M fields
            // event queue can send multiple responses for large requests
            foreach (BBCOMM.Message message in response.GetMessages())
            {
                // extract N securities
                BBCOMM.Element securities = message.GetElement(SECURITY_DATA);
                int nSecurities = securities.NumValues;
                //
                // loop through all securities
                for (int i = 0; i < nSecurities; i++)
                {
                    // extract one security and fields for this security
                    BBCOMM.Element security = securities.GetValueAsElement(i);
                    BBCOMM.Element fields = security.GetElement(FIELD_DATA);
                    int sequenceNumber = security.GetElementAsInt32(SEQUENCE_NUMBER);
                    int nFieldNames = fieldNames.Count;
                    //
                    // loop through all M fields for this security
                    for (int j = 0; j < nFieldNames; j++)
                    {
                        // if the requested field has been found, pack value into result data structure
                        if (fields.HasElement(fieldNames[j]))
                        {
                            result[sequenceNumber, 0, j] = fields.GetElement(fieldNames[j]).GetValue();
                        }
                        // otherwise, pack NOT_AVAILABLE string into data structure
                        else
                        {
                            result[sequenceNumber, 0, j] = NOT_AVAILABLE;
                        }
                    }
                }
            }
        }
    }
    //
    // concrete class implementation for processing historical data request
    public class HistoricalDataRequest : BBCOMMDataRequest
    {
        private bool hasDimensions = false;
        //
        // optional parameters are configured to retrieve time-series having actual daily observations, including all weekdays,
        // in the case of non-trading days the previous date value will be used.
        public HistoricalDataRequest(List<string> bloombergSecurityNames, List<string> bloombergFieldNames,
            DateTime bloombergStartDate, DateTime BloombergEndDate,
            E_PRICING_OPTION bloombergPricingOption = E_PRICING_OPTION.PRICING_OPTION_PRICE,
            E_PERIODICITY_SELECTION bloombergPeriodicitySelection = E_PERIODICITY_SELECTION.DAILY,
            E_PERIODICITY_ADJUSTMENT bloombergPeriodicityAdjustment = E_PERIODICITY_ADJUSTMENT.ACTUAL,
            E_NON_TRADING_DAY_FILL_OPTION bloombergNonTradingDayFillOption = E_NON_TRADING_DAY_FILL_OPTION.ALL_CALENDAR_DAYS,
            E_NON_TRADING_DAY_FILL_METHOD bloombergNonTradingDayFillMethod = E_NON_TRADING_DAY_FILL_METHOD.PREVIOUS_VALUE,
            string bloombergOverrideCurrency = "")
        {
            // ctor : create historical data request without field overrides
            requestType = REQUEST_TYPE_HISTORICAL;
            securityNames = bloombergSecurityNames;
            fieldNames = bloombergFieldNames;
            startDate = bloombergStartDate.ToString(BLOOMBERG_DATE_FORMAT);
            endDate = BloombergEndDate.ToString(BLOOMBERG_DATE_FORMAT);
            //
            pricingOption = bloombergPricingOption;
            periodicitySelection = bloombergPeriodicitySelection;
            periodicityAdjustment = bloombergPeriodicityAdjustment;
            nonTradingDayFillOption = bloombergNonTradingDayFillOption;
            nonTradingDayFillMethod = bloombergNonTradingDayFillMethod;
            overrideCurrency = bloombergOverrideCurrency;
        }
        public HistoricalDataRequest(List<string> bloombergSecurityNames, List<string> bloombergFieldNames,
            DateTime bloombergStartDate, DateTime BloombergEndDate, List<string> bloombergOverrideFields,
            List<string> bloombergOverrideValues,
            E_PRICING_OPTION bloombergPricingOption = E_PRICING_OPTION.PRICING_OPTION_PRICE,
            E_PERIODICITY_SELECTION bloombergPeriodicitySelection = E_PERIODICITY_SELECTION.DAILY,
            E_PERIODICITY_ADJUSTMENT bloombergPeriodicityAdjustment = E_PERIODICITY_ADJUSTMENT.ACTUAL,
            E_NON_TRADING_DAY_FILL_OPTION bloombergNonTradingDayFillOption = E_NON_TRADING_DAY_FILL_OPTION.ALL_CALENDAR_DAYS,
            E_NON_TRADING_DAY_FILL_METHOD bloombergNonTradingDayFillMethod = E_NON_TRADING_DAY_FILL_METHOD.PREVIOUS_VALUE,
            string bloombergOverrideCurrency = "")
        {
            // ctor : create historical data request with field overrides
            requestType = REQUEST_TYPE_HISTORICAL;
            securityNames = bloombergSecurityNames;
            fieldNames = bloombergFieldNames;
            overrideFields = bloombergOverrideFields;
            overrideValues = bloombergOverrideValues;
            startDate = bloombergStartDate.ToString(BLOOMBERG_DATE_FORMAT);
            endDate = BloombergEndDate.ToString(BLOOMBERG_DATE_FORMAT);
            //
            pricingOption = bloombergPricingOption;
            periodicitySelection = bloombergPeriodicitySelection;
            periodicityAdjustment = bloombergPeriodicityAdjustment;
            nonTradingDayFillOption = bloombergNonTradingDayFillOption;
            nonTradingDayFillMethod = bloombergNonTradingDayFillMethod;
            overrideCurrency = bloombergOverrideCurrency;
        }
        protected override void ProcessDataResponse(ref BBCOMM.Event response)
        {
            // unzip and pack messages received from BBCOMM server
            // receive one security per message and multiple messages per event
            foreach (BBCOMM.Message message in response.GetMessages())
            {
                // extract security and fields
                BBCOMM.Element security = message.GetElement(SECURITY_DATA);
                BBCOMM.Element fields = security.GetElement(FIELD_DATA);
                //
                int sequenceNumber = security.GetElementAsInt32(SEQUENCE_NUMBER);
                int nFieldNames = fieldNames.Count;
                int nObservationDates = fields.NumValues;
                //
                // the exact dimension will be known only, when the response has been received from BBCOMM server
                if (!hasDimensions)
                {
                    // define result data structure dimensions for historical data request
                    // observation date will be stored into first field for each observation date
                    result = new dynamic[securityNames.Count, nObservationDates, fieldNames.Count + 1];
                    hasDimensions = true;
                }
                //
                // loop through all observation dates
                for (int i = 0; i < nObservationDates; i++)
                {
                    // extract all field data for a single observation date
                    BBCOMM.Element observationDateFields = fields.GetValueAsElement(i);
                    //
                    // pack observation date into data structure
                    result[sequenceNumber, i, 0] = observationDateFields.GetElementAsDatetime(DATE);
                    //
                    // then, loop through all 'user-requested' fields for a given observation date
                    // and pack results data into data structure
                    for (int j = 0; j < nFieldNames; j++)
                    {
                        // pack field value into data structure if such value has been found
                        if (observationDateFields.HasElement(fieldNames[j]))
                        {
                            result[sequenceNumber, i, j + 1] = observationDateFields.GetElement(fieldNames[j]).GetValue();
                        }
                        // otherwise, pack NOT_AVAILABLE string into data structure
                        else
                        {
                            result[sequenceNumber, i, j + 1] = NOT_AVAILABLE;
                        }
                    }
                }
            }
        }
    }
}

AFTERTHOUGHTS

Stripped from all of its superficial complexities, this example program is actually nothing more, but very standard INPUT DATA - PROCESS DATA - OUTPUT RESULTS scheme. However, by implementing some useful Design Patterns (Builder, Factory method, Visitor), we are now having a lot of flexibility and configurability in this program. Getting results file with all requested fields from Bloomberg, for any new portfolio (for a list of Bloomberg Swap Manager transactions stored in a file) is just a matter of creating a new configuration file into existing repository folder.

One small disappointing issue is the manual administration tasks of hosting that list of transactions which are stored in Bloomberg Swap Manager. So far, I have not been able to find a way to request those transactions directly from Bloomberg. However, Bloomberg has recently been releasing its Swaps Toolkit for Excel, which (according to our Bloomberg account manager) has also programming interface. I might get some new ideas for handling swaps books, as soon as I am familiar enough with the product.

UML class diagram has been created with the online tool called YUML. It is completely free, easy to use, the most flexible and the coolest UML tool what I have been using so far, so check it out. And thanks again for spending your precious time here and reading my blog.

-Mike