Monday, April 6, 2015

NAG Copula wrapper for C#

I've got you under my skin, Frank Sinatra is singing in computer radio. Somehow, it reminds me about my obsession with Copula, what I have been chewing in a form or another, again and again for some time already. So, this time I wanted to share the latest fruit of this obsession as NAG Copula wrapper for C#.

NAG G05 Copula wrapper

With wrapper, one can create correlated random numbers
  • from any distribution (assuming appropriate inverse transformation method is provided)
  • by using Gaussian or Student Copula
The first task, creation of correlated random numbers inside NAGCopula class, is performed by NAG G05 library methods g05rc (Student) and g05rd (Gaussian). The result provided by both of these methods, is a matrix filled with correlated uniform random numbers. The second task, mapping correlated uniform random numbers back to a desired probability distribution, is performed by InverseTransformation class implementation inside NAGCopula class.

Client can create NAGCopula object with two different constructors, with or without inverse transformation. Also, NAGCopula model selection (Gaussian, Student) will be made, based on user-given value for degrees-of-freedom parameter. This parameter, being greater than zero, will automatically trigger the use of Student Copula model.

NAGCopula class, InverseTransformation class hierarchy and Client tester program is presented below. Program will first retrieve a matrix filled with correlated random numbers from NAGCopula object and then writing the content of that matrix into a given text file. Define new addresses for file outputs, if needed. Create a new C# console project, copyPaste the whole content into a new cs file and run the program.

Copula outputs

The results of one client program run is presented in the table below. For a given bi-variate test scheme, the both Copula models (Gaussian, Student) were used to create 5000 correlated random number pairs. The created random numbers were also transformed back to Standard Normal and Exponential distributions.




















The Program


using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using NagLibrary;
using System.IO;

namespace NAGCopulaNamespace
{
    // CLIENT
    class Program
    {
        static void Main(string[] args)
        {
            try
            {
                double[,] covariance = CreateCovarianceMatrix();
                int[] seed = new int[1];
                int n = 5000;
                double[,] randomMatrix;
                //
                // create correlated uniform random numbers using gaussian copula
                NAGCopula gaussian = new NAGCopula(covariance, 1, 1, 2);
                seed[0] = Math.Abs(Guid.NewGuid().GetHashCode());
                randomMatrix = gaussian.Create(n, seed);
                Print(@"C:\temp\GaussianCopulaNumbers.txt", randomMatrix);
                //
                // create correlated uniform random numbers using student copula
                NAGCopula student = new NAGCopula(covariance, 1, 1, 2, 3);
                seed[0] = Math.Abs(Guid.NewGuid().GetHashCode());
                randomMatrix = student.Create(n, seed);
                Print(@"C:\temp\StudentCopulaNumbers.txt", randomMatrix);
                //
                // create correlated standard normal random numbers using gaussian copula
                NAGCopula gaussianTransformed = new NAGCopula(covariance, 1, 1, 2, new StandardNormal());
                seed[0] = Math.Abs(Guid.NewGuid().GetHashCode());
                randomMatrix = gaussianTransformed.Create(n, seed);
                Print(@"C:\temp\GaussianTransformedCopulaNumbers.txt", randomMatrix);
                //
                // create correlated standard normal random numbers using student copula
                NAGCopula studentTransformed = new NAGCopula(covariance, 1, 1, 2, new StandardNormal(), 3);
                seed[0] = Math.Abs(Guid.NewGuid().GetHashCode());
                randomMatrix = studentTransformed.Create(n, seed);
                Print(@"C:\temp\StudentTransformedCopulaNumbers.txt", randomMatrix);
                //
                // create correlated exponential random numbers using gaussian copula
                NAGCopula gaussianTransformed2 = new NAGCopula(covariance, 1, 1, 2, new Exponential(0.078));
                seed[0] = Math.Abs(Guid.NewGuid().GetHashCode());
                randomMatrix = gaussianTransformed2.Create(n, seed);
                Print(@"C:\temp\GaussianTransformedCopulaNumbers2.txt", randomMatrix);
                //
                // create correlated exponential random numbers using student copula
                NAGCopula studentTransformed2 = new NAGCopula(covariance, 1, 1, 2, new Exponential(0.078), 3);
                seed[0] = Math.Abs(Guid.NewGuid().GetHashCode());
                randomMatrix = studentTransformed2.Create(n, seed);
                Print(@"C:\temp\StudentTransformedCopulaNumbers2.txt", randomMatrix);
            }
            catch (Exception e)
            {
                Console.WriteLine(e.Message);
            }
        }
        static void Print(string filePathName, double[,] arr)
        {
            // write generated correlated random numbers into text file
            StreamWriter writer = new StreamWriter(filePathName);
            //
            for (int i = 0; i < arr.GetLength(0); i++)
            {
                for (int j = 0; j < arr.GetLength(1); j++)
                {
                    writer.Write("{0};", arr[i, j].ToString().Replace(',','.'));
                }
                writer.WriteLine();
            }
            writer.Close();
        }
        static double[,] CreateCovarianceMatrix()
        {
            // hard-coded data for 2x2 co-variance matrix
            // correlation is (roughly) 0.75
            double[,] c = new double[2, 2];
            c[0, 0] = 0.00137081575700264; c[0, 1] = 0.00111385579983184;
            c[1, 0] = 0.00111385579983184; c[1, 1] = 0.00161530858115086;
            return c;
        }
    }
    //
    //
    // NAG COPULA WRAPPER
    public class NAGCopula
    {
        private InverseTransformation transformer;
        private double[,] covariance;
        private int genID;
        private int subGenID;
        private int mode;
        private int df;
        private int m;
        private int errorNumber;
        private double[] r;
        private double[,] result;
        //
        public NAGCopula(double[,] covariance, int genID,
            int subGenID, int mode, int df = 0)
        {
            // ctor : create correlated uniform random numbers
            // degrees-of-freedom parameter (df), being greater than zero, 
            // will automatically trigger the use of student copula
            this.covariance = covariance;
            this.genID = genID;
            this.subGenID = subGenID;
            this.mode = mode;
            this.df = df;
            this.m = covariance.GetLength(1);
            r = new double[m * (m + 1) + (df > 0 ? 2 : 1)];
        }
        public NAGCopula(double[,] covariance, int genID, int subGenID,
            int mode, InverseTransformation transformer, int df = 0)
        {
            // ctor : create correlated random numbers from distribution X
            // degrees-of-freedom parameter (df), being greater than zero, 
            // will automatically trigger the use of student copula
            this.transformer = transformer;
            this.covariance = covariance;
            this.genID = genID;
            this.subGenID = subGenID;
            this.mode = mode;
            this.df = df;
            this.m = covariance.GetLength(1);
            r = new double[m * (m + 1) + (df > 0 ? 2 : 1)];
        }
        public double[,] Create(int n, int[] seed)
        {
            result = new double[n, m];
            //
            G05.G05State g05State = new G05.G05State(genID, subGenID, seed, out errorNumber);
            if (errorNumber != 0) throw new Exception("G05 state failure");
            //
            if (this.df != 0)
            {
                // student copula
                G05.g05rc(mode, n, df, m, covariance, r, g05State, result, out errorNumber);
            }
            else
            {
                // gaussian copula
                G05.g05rd(mode, n, m, covariance, r, g05State, result, out errorNumber);
            }
            //
            if (errorNumber != 0) throw new Exception("G05 method failure");
            if (transformer != null) transform();
            return result;
        }
        private void transform()
        {
            // map uniform random numbers to distribution X
            for (int i = 0; i < result.GetLength(0); i++)
            {
                for (int j = 0; j < result.GetLength(1); j++)
                {
                    result[i, j] = transformer.Inverse(result[i, j]);
                }
            }
        }
    }
    //
    //
    // INVERSE TRANSFORMATION LIBRARY
    // base class for all inverse transformation classes
    public abstract class InverseTransformation
    {
        public abstract double Inverse(double x);
    }
    //
    // mapping uniform random variate to exponential distribution
    public class Exponential : InverseTransformation
    {
        private double lambda;
        public Exponential(double lambda)
        {
            this.lambda = lambda;
        }
        public override double Inverse(double x)
        {
            return Math.Pow(-lambda, -1.0) * Math.Log(x);
        }
    }
    //
    // mapping uniform random variate to standard normal distribution
    public class StandardNormal : InverseTransformation
    {
        public override double Inverse(double x)
        {
            const double a1 = -39.6968302866538; const double a2 = 220.946098424521;
            const double a3 = -275.928510446969; const double a4 = 138.357751867269;
            const double a5 = -30.6647980661472; const double a6 = 2.50662827745924;
            //
            const double b1 = -54.4760987982241; const double b2 = 161.585836858041;
            const double b3 = -155.698979859887; const double b4 = 66.8013118877197;
            const double b5 = -13.2806815528857;
            //
            const double c1 = -7.78489400243029E-03; const double c2 = -0.322396458041136;
            const double c3 = -2.40075827716184; const double c4 = -2.54973253934373;
            const double c5 = 4.37466414146497; const double c6 = 2.93816398269878;
            //
            const double d1 = 7.78469570904146E-03; const double d2 = 0.32246712907004;
            const double d3 = 2.445134137143; const double d4 = 3.75440866190742;
            //
            const double p_low = 0.02425; const double p_high = 1.0 - p_low;
            double q = 0.0; double r = 0.0; double c = 0.0;
            //
            if ((x > 0.0) && (x < 1.0))
            {
                if (x < p_low)
                {
                    // rational approximation for lower region
                    q = Math.Sqrt(-2.0 * Math.Log(x));
                    c = (((((c1 * q + c2) * q + c3) * q + c4) * q + c5) * q + c6)
                        / ((((d1 * q + d2) * q + d3) * q + d4) * q + 1);
                }
                if ((x >= p_low) && (x <= p_high))
                {
                    // rational approximation for middle region
                    q = x - 0.5; r = q * q;
                    c = (((((a1 * r + a2) * r + a3) * r + a4) * r + a5) * r + a6) * q
                        / (((((b1 * r + b2) * r + b3) * r + b4) * r + b5) * r + 1);
                }
                if (x > p_high)
                {
                    // rational approximation for upper region
                    q = Math.Sqrt(-2 * Math.Log(1 - x));
                    c = -(((((c1 * q + c2) * q + c3) * q + c4) * q + c5) * q + c6)
                        / ((((d1 * q + d2) * q + d3) * q + d4) * q + 1);
                }
            }
            else
            {
                // throw if given x value is out of bounds
                // (less or equal to 0.0, greater or equal to 1.0)
                throw new Exception("input parameter out of bounds");
            }
            return c;
        }
    }
}


Afterthoughts

It is quite easy to appreciate the numerical work performed by NAG G05 library. In the case you might be interested about how to create non-correlated random numbers with NAG, check out my blog post in here. If you like the stuff, you can get yourself more familiar with NAG libraries in here. So, that was all I wanted to share with you again. Thanks for reading my blog. Good night. Mike.

Saturday, March 21, 2015

Bloomberg API Wrapper for C#

As a starter for the year 2015, I wanted to share the current version of my C# wrapper, which can be used to execute reference and historical data requests for Bloomberg BBCOMM server.

In the past, I have been posting all updates for corresponding VBA wrapper in separate blog posts and I think some people have been finding this procedure to be at least inconvinient. To be better prepared for those program modifications and bug fixes which will be found out in the future, I will sanctify this one blog post for all postings concerning Bloomberg API Wrapper class.

Release date : 21-Mar-2015

Client program can use this wrapper to execute reference and historical data requests for BBCOMM server. Design of the wrapper class is simple. We have abstract base class BBCOMMDataRequest, which is holding all the required constants, enumerators, BBCOMM objects and data structures. Base class is implementing most of the required functionality, such as
  • Creating and starting BBCOMM session
  • Construction of Request object
  • Sending Request object to BBCOMM server
  • Closing BBCOMM session
For all of the functionalities described above, one can easily find well-written documentations to work with from Bloomberg. However, one of the values what I am trying to add with this wrapper is the one, which is always missing in these otherwise great documentations: how to pack all requested data into data structures to be easily used by the client?

After a few design iterations and brainstorming moments, I concluded that the program is actually the same for both reference and historical data requests for the most parts. Only relevant difference is algorithm implementation for iterating BBCOMM response and packing values into result data structure. After some further woodshedding, I finally decided to create the design, in which the base class is handling all the common functionalities and derived classes (ReferenceDataRequest, HistoricalDataRequest) are handling the specific algorithm implementation for iterating and packing. Client will have an access to constructors of derived classes, in which securities, fields, overrides, dates and all the other optional parameters are given in for wrapper.

Client will create dynamically-typed 3-dimensional array, into which the resulting data will be assigned by ProcessData method of wrapper. More specifically, for

  • Reference data request : result[1, 0, 1] = 2nd security, 2nd field
  • Historical data request : result[3, 1, 2] = 4th security, 2nd observation date, 3rd field. For the third dimension of the array, index value zero will always store the observation date

All relevant examples can be found in tester program presented below along with the wrapper. To be able to run tester program, one has to create a new console project and copyPaste the program given below. Moreover, project must have reference to Bloomberglp.Blpapi.dll file. For further development and testing purposes, you can use the actual Bloomberg terminal or check out Bloomberg API Emulator.

That's all for now. Thanks for reading my blog. Mike.



Wrapper : 21-Mar-2015

using System;
using System.Collections.Generic;
using System.Linq;
using BBCOMM = Bloomberglp.Blpapi;
using BloombergAPI;
//
// CLIENT PROGRAM
namespace TesterProgram
{
    class Program
    {
        // data structures
        static List<string> bloombergSecurityNames;
        static List<string> bloombergFieldNames;
        static List<string> bloombergOverrideFields;
        static List<string> bloombergOverrideValues;
        static dynamic[, ,] result;
        //
        static int Main()
        {
            try
            {
                Console.WriteLine("CASE 1 : create reference data request without overrides >");
                Console.WriteLine();
                //
                // create securities
                bloombergSecurityNames = new List<string>();
                bloombergSecurityNames.Add("GOOG US Equity");
                bloombergSecurityNames.Add("CSCO US Equity");
                bloombergSecurityNames.Add("CCE US Equity");
                //
                // create fields
                bloombergFieldNames = new List<string>();
                bloombergFieldNames.Add("PX_LAST");
                bloombergFieldNames.Add("BEST_EPS");
                bloombergFieldNames.Add("GICS_SUB_INDUSTRY_NAME");
                bloombergFieldNames.Add("EQY_OPT_AVAIL");
                bloombergFieldNames.Add("LATEST_ANN_DT_ANNUAL");
                bloombergFieldNames.Add("PX_ROUND_LOT_SIZE");
                bloombergFieldNames.Add("MIKE_JUNIPERHILL_SHOE_SIZE");
                //
                // create wrapper object, retrieve and print data
                BBCOMMDataRequest wrapper = new ReferenceDataRequest(bloombergSecurityNames, bloombergFieldNames);
                result = wrapper.ProcessData();
                Print(bloombergSecurityNames, result);
                //
                Console.WriteLine("CASE 2 : create reference data request with override >");
                Console.WriteLine();
                //
                // re-create fields
                bloombergFieldNames = new List<string>();
                bloombergFieldNames.Add("PX_LAST");
                bloombergFieldNames.Add("BEST_EPS");
                //
                // create override for best fiscal period
                bloombergOverrideFields = new List<string>();
                bloombergOverrideFields.Add("BEST_FPERIOD_OVERRIDE");
                bloombergOverrideValues = new List<string>();
                bloombergOverrideValues.Add("2FY");
                //
                // retrieve and print data
                wrapper = new ReferenceDataRequest(bloombergSecurityNames, bloombergFieldNames, bloombergOverrideFields, bloombergOverrideValues);
                result = wrapper.ProcessData();
                Print(bloombergSecurityNames, result);
                //
                Console.WriteLine("CASE 3 : create historical data request for one security >");
                Console.WriteLine();
                //
                bloombergSecurityNames = new List<string>();
                bloombergSecurityNames.Add("GOOG US Equity");
                //
                // re-create field
                bloombergFieldNames = new List<string>();
                bloombergFieldNames.Add("PX_LAST");
                //
                // create dates
                DateTime startDate = DateTime.Today.AddDays((double)-21);
                DateTime endDate = DateTime.Today.AddDays((double)-1);
                //
                // retrieve and print data
                // request time-series for Google : actual daily frequency, but only for weekdays and converted to Thai baht
                wrapper = new HistoricalDataRequest(bloombergSecurityNames, bloombergFieldNames, startDate, endDate,
                    bloombergNonTradingDayFillOption: E_NON_TRADING_DAY_FILL_OPTION.NON_TRADING_WEEKDAYS,
                    bloombergOverrideCurrency: "THB");
                //
                result = wrapper.ProcessData();
                Print(bloombergSecurityNames, result);
                //
                Console.WriteLine("CASE 4 : create historical data request for three securities >");
                Console.WriteLine();
                //
                // re-create securities
                bloombergSecurityNames = new List<string>();
                bloombergSecurityNames.Add("GOOG US Equity");
                bloombergSecurityNames.Add("CSCO US Equity");
                bloombergSecurityNames.Add("CCE US Equity");
                //
                // re-create field
                bloombergFieldNames = new List<string>();
                bloombergFieldNames.Add("PX_LAST");
                //
                // retrieve and print data
                // request three time-series : use default settings to retrieve date-consistent result data
                wrapper = new HistoricalDataRequest(bloombergSecurityNames, bloombergFieldNames, startDate, endDate);
                result = wrapper.ProcessData();
                Print(bloombergSecurityNames, result);
            }
            catch (Exception e)
            {
                Console.WriteLine(e.Message);
            }
            //
            Console.ReadLine();
            return 0;
        }
        // result printer
        public static void Print(List<string> securities, dynamic[, ,] data)
        {
            for (int i = 0; i < data.GetLength(0); i++)
            {
                Console.WriteLine(securities[i]);
                for (int j = 0; j < data.GetLength(1); j++)
                {
                    for (int k = 0; k < data.GetLength(2); k++)
                    {
                        Console.WriteLine(data[i, j, k]);
                    }
                }
                Console.WriteLine();
            }
        }
    }
}
//
// WRAPPER
namespace BloombergAPI
{
    // enumerators for historical data request settings
    public enum E_PRICING_OPTION { PRICING_OPTION_PRICE, PRICING_OPTION_YIELD };
    public enum E_PERIODICITY_ADJUSTMENT { ACTUAL, CALENDAR, FISCAL };
    public enum E_PERIODICITY_SELECTION { DAILY, WEEKLY, MONTHLY, QUARTERLY, SEMI_ANNUALLY, YEARLY };
    public enum E_NON_TRADING_DAY_FILL_OPTION { NON_TRADING_WEEKDAYS, ALL_CALENDAR_DAYS, ACTIVE_DAYS_ONLY };
    public enum E_NON_TRADING_DAY_FILL_METHOD { PREVIOUS_VALUE, NIL_VALUE };
    //
    // abstract base class for data request
    public abstract class BBCOMMDataRequest
    {
        // BBCOMM names
        protected readonly BBCOMM.Name SECURITY_DATA = new BBCOMM.Name("securityData");
        protected readonly BBCOMM.Name FIELD_DATA = new BBCOMM.Name("fieldData");
        protected readonly BBCOMM.Name FIELD_ID = new BBCOMM.Name("fieldId");
        protected readonly BBCOMM.Name VALUE = new BBCOMM.Name("value");
        protected readonly BBCOMM.Name OVERRIDES = new BBCOMM.Name("overrides");
        protected readonly BBCOMM.Name SECURITIES = new BBCOMM.Name("securities");
        protected readonly BBCOMM.Name FIELDS = new BBCOMM.Name("fields");
        protected readonly BBCOMM.Name SEQUENCE_NUMBER = new BBCOMM.Name("sequenceNumber");
        protected readonly BBCOMM.Name START_DATE = new BBCOMM.Name("startDate");
        protected readonly BBCOMM.Name END_DATE = new BBCOMM.Name("endDate");
        protected readonly BBCOMM.Name DATE = new BBCOMM.Name("date");
        protected readonly BBCOMM.Name PRICING_OPTION = new BBCOMM.Name("pricingOption");
        protected readonly BBCOMM.Name PERIODICITY_ADJUSTMENT = new BBCOMM.Name("periodicityAdjustment");
        protected readonly BBCOMM.Name PERIODICITY_SELECTION = new BBCOMM.Name("periodicitySelection");
        protected readonly BBCOMM.Name NON_TRADING_DAY_FILL_OPTION = new BBCOMM.Name("nonTradingDayFillOption");
        protected readonly BBCOMM.Name NON_TRADING_DAY_FILL_METHOD = new BBCOMM.Name("nonTradingDayFillMethod");
        protected readonly BBCOMM.Name OVERRIDE_CURRENCY = new BBCOMM.Name("currency");
        //
        // const strings, enumerators, etc.
        protected readonly string NOT_AVAILABLE = "#N/A";
        protected readonly string SESSION_EXCEPTION = "Session not started";
        protected readonly string SERVICE_EXCEPTION = "Service not opened";
        protected readonly string REQUEST_TYPE_REFERENCE = "ReferenceDataRequest";
        protected readonly string REQUEST_TYPE_HISTORICAL = "HistoricalDataRequest";
        protected readonly string REFERENCE_DATA_SERVICE = "//blp/refdata";
        protected readonly string BLOOMBERG_DATE_FORMAT = "yyyyMMdd";
        protected E_PRICING_OPTION pricingOption;
        protected E_PERIODICITY_ADJUSTMENT periodicityAdjustment;
        protected E_PERIODICITY_SELECTION periodicitySelection;
        protected E_NON_TRADING_DAY_FILL_OPTION nonTradingDayFillOption;
        protected E_NON_TRADING_DAY_FILL_METHOD nonTradingDayFillMethod;
        protected string requestType;
        protected string startDate;
        protected string endDate;
        protected string overrideCurrency;
        //
        // wrapped BBCOMM objects
        protected BBCOMM.Session session;
        protected BBCOMM.Service service;
        protected BBCOMM.Request request;
        //
        // input data structures
        protected List<string> securityNames = new List<string>();
        protected List<string> fieldNames = new List<string>();
        protected List<string> overrideFields = new List<string>();
        protected List<string> overrideValues = new List<string>();
        //
        // output result data structure
        protected dynamic[, ,] result;
        //
        public dynamic[, ,] ProcessData()
        {
            Open();
            CreateRequest();
            SendRequest();
            Close();
            return result;
        }
        private void Open()
        {
            // create and start bloomberg BBCOMM session
            BBCOMM.SessionOptions sessionOptions = new BBCOMM.SessionOptions();
            session = new BBCOMM.Session(sessionOptions);
            if (!session.Start()) throw new Exception(SESSION_EXCEPTION);
            //
            // get service from session object and create request by service object
            if (!session.OpenService(REFERENCE_DATA_SERVICE)) throw new Exception(SERVICE_EXCEPTION);
            service = session.GetService(REFERENCE_DATA_SERVICE);
            request = service.CreateRequest(requestType);
        }
        private void CreateRequest()
        {
            // append securities, fields
            foreach (string securityName in securityNames) request.Append(SECURITIES, securityName);
            foreach (string fieldName in fieldNames) request.Append(FIELDS, fieldName);
            //
            // conditionally, append overrides into request object
            if (overrideFields.Count > 0)
            {
                BBCOMM.Element requestOverrides = request.GetElement(OVERRIDES);
                for (int i = 0; i < overrideFields.Count; i++)
                {
                    BBCOMM.Element requestOverride = requestOverrides.AppendElement();
                    requestOverride.SetElement(FIELD_ID, overrideFields[i]);
                    requestOverride.SetElement(VALUE, overrideValues[i]);
                }
            }
            // set optional parameters for historical data request
            if (requestType == REQUEST_TYPE_HISTORICAL)
            {
                request.Set(START_DATE, startDate);
                request.Set(END_DATE, endDate);
                request.Set(PRICING_OPTION, pricingOption.ToString());
                request.Set(PERIODICITY_ADJUSTMENT, periodicityAdjustment.ToString());
                request.Set(PERIODICITY_SELECTION, periodicitySelection.ToString());
                request.Set(NON_TRADING_DAY_FILL_OPTION, nonTradingDayFillOption.ToString());
                request.Set(NON_TRADING_DAY_FILL_METHOD, nonTradingDayFillMethod.ToString());
                if (overrideCurrency != String.Empty) request.Set(OVERRIDE_CURRENCY, overrideCurrency);
            }
        }
        private void SendRequest()
        {
            // send constructed request to BBCOMM server
            long ID = Guid.NewGuid().GetHashCode();
            session.SendRequest(request, new BBCOMM.CorrelationID(ID));
            bool isProcessing = true;
            //
            while (isProcessing)
            {
                // receive data response from BBCOMM server, send 
                // response to be processed by sub-classed algorithm
                BBCOMM.Event response = session.NextEvent();
                switch (response.Type)
                {
                    case BBCOMM.Event.EventType.PARTIAL_RESPONSE:
                        ProcessDataResponse(ref response);
                        break;
                    case BBCOMM.Event.EventType.RESPONSE:
                        ProcessDataResponse(ref response);
                        isProcessing = false;
                        break;
                    default:
                        break;
                }
            }
        }
        private void Close()
        {
            // close BBCOMM session
            if (session != null) session.Stop();
        }
        //
        // sub-classes are providing specific algorithm implementations for 
        // processing and packing BBCOMM server response data into resulting data structure
        protected abstract void ProcessDataResponse(ref BBCOMM.Event response);
    }
    //
    // concrete class implementation for processing reference data request
    public class ReferenceDataRequest : BBCOMMDataRequest
    {
        public ReferenceDataRequest(List<string> bloombergSecurityNames,
            List<string> bloombergFieldNames)
        {
            // ctor : create reference data request without field overrides
            requestType = REQUEST_TYPE_REFERENCE;
            securityNames = bloombergSecurityNames;
            fieldNames = bloombergFieldNames;
            //
            // define result data structure dimensions for reference data request
            result = new dynamic[securityNames.Count, 1, fieldNames.Count];
        }
        public ReferenceDataRequest(List<string> bloombergSecurityNames,
            List<string> bloombergFieldNames, List<string> bloombergOverrideFields,
            List<string> bloombergOverrideValues)
        {
            // ctor : create reference data request with field overrides
            requestType = REQUEST_TYPE_REFERENCE;
            securityNames = bloombergSecurityNames;
            fieldNames = bloombergFieldNames;
            overrideFields = bloombergOverrideFields;
            overrideValues = bloombergOverrideValues;
            //
            // define result data structure dimensions for reference data request
            result = new dynamic[securityNames.Count, 1, fieldNames.Count];
        }
        protected override void ProcessDataResponse(ref BBCOMM.Event response)
        {
            // receive response, which contains N securities and M fields
            // event queue can send multiple responses for large requests
            foreach (BBCOMM.Message message in response.GetMessages())
            {
                // extract N securities
                BBCOMM.Element securities = message.GetElement(SECURITY_DATA);
                int nSecurities = securities.NumValues;
                //
                // loop through all securities
                for (int i = 0; i < nSecurities; i++)
                {
                    // extract one security and fields for this security
                    BBCOMM.Element security = securities.GetValueAsElement(i);
                    BBCOMM.Element fields = security.GetElement(FIELD_DATA);
                    int sequenceNumber = security.GetElementAsInt32(SEQUENCE_NUMBER);
                    int nFieldNames = fieldNames.Count;
                    //
                    // loop through all M fields for this security
                    for (int j = 0; j < nFieldNames; j++)
                    {
                        // if the requested field has been found, pack value into result data structure
                        if (fields.HasElement(fieldNames[j]))
                        {
                            result[sequenceNumber, 0, j] = fields.GetElement(fieldNames[j]).GetValue();
                        }
                        // otherwise, pack NOT_AVAILABLE string into data structure
                        else
                        {
                            result[sequenceNumber, 0, j] = NOT_AVAILABLE;
                        }
                    }
                }
            }
        }
    }
    //
    // concrete class implementation for processing historical data request
    public class HistoricalDataRequest : BBCOMMDataRequest
    {
        private bool hasDimensions = false;
        //
        // optional parameters are configured to retrieve time-series having actual daily observations, including all weekdays,
        // in the case of non-trading days the previous date value will be used.
        public HistoricalDataRequest(List<string> bloombergSecurityNames, List<string> bloombergFieldNames, 
            DateTime bloombergStartDate, DateTime BloombergEndDate, 
            E_PRICING_OPTION bloombergPricingOption = E_PRICING_OPTION.PRICING_OPTION_PRICE,
            E_PERIODICITY_SELECTION bloombergPeriodicitySelection = E_PERIODICITY_SELECTION.DAILY, 
            E_PERIODICITY_ADJUSTMENT bloombergPeriodicityAdjustment = E_PERIODICITY_ADJUSTMENT.ACTUAL,
            E_NON_TRADING_DAY_FILL_OPTION bloombergNonTradingDayFillOption = E_NON_TRADING_DAY_FILL_OPTION.ALL_CALENDAR_DAYS, 
            E_NON_TRADING_DAY_FILL_METHOD bloombergNonTradingDayFillMethod = E_NON_TRADING_DAY_FILL_METHOD.PREVIOUS_VALUE,
            string bloombergOverrideCurrency = "")
        {
            // ctor : create historical data request without field overrides
            requestType = REQUEST_TYPE_HISTORICAL;
            securityNames = bloombergSecurityNames;
            fieldNames = bloombergFieldNames;
            startDate = bloombergStartDate.ToString(BLOOMBERG_DATE_FORMAT);
            endDate = BloombergEndDate.ToString(BLOOMBERG_DATE_FORMAT);
            //
            pricingOption = bloombergPricingOption;
            periodicitySelection = bloombergPeriodicitySelection;
            periodicityAdjustment = bloombergPeriodicityAdjustment;
            nonTradingDayFillOption = bloombergNonTradingDayFillOption;
            nonTradingDayFillMethod = bloombergNonTradingDayFillMethod;
            overrideCurrency = bloombergOverrideCurrency;
        }
        public HistoricalDataRequest(List<string> bloombergSecurityNames, List<string> bloombergFieldNames,
            DateTime bloombergStartDate, DateTime BloombergEndDate, List<string> bloombergOverrideFields,
            List<string> bloombergOverrideValues,
            E_PRICING_OPTION bloombergPricingOption = E_PRICING_OPTION.PRICING_OPTION_PRICE,
            E_PERIODICITY_SELECTION bloombergPeriodicitySelection = E_PERIODICITY_SELECTION.DAILY,
            E_PERIODICITY_ADJUSTMENT bloombergPeriodicityAdjustment = E_PERIODICITY_ADJUSTMENT.ACTUAL,
            E_NON_TRADING_DAY_FILL_OPTION bloombergNonTradingDayFillOption = E_NON_TRADING_DAY_FILL_OPTION.ALL_CALENDAR_DAYS,
            E_NON_TRADING_DAY_FILL_METHOD bloombergNonTradingDayFillMethod = E_NON_TRADING_DAY_FILL_METHOD.PREVIOUS_VALUE,
            string bloombergOverrideCurrency = "")
        {
            // ctor : create historical data request with field overrides
            requestType = REQUEST_TYPE_HISTORICAL;
            securityNames = bloombergSecurityNames;
            fieldNames = bloombergFieldNames;
            overrideFields = bloombergOverrideFields;
            overrideValues = bloombergOverrideValues;
            startDate = bloombergStartDate.ToString(BLOOMBERG_DATE_FORMAT);
            endDate = BloombergEndDate.ToString(BLOOMBERG_DATE_FORMAT);
            //
            pricingOption = bloombergPricingOption;
            periodicitySelection = bloombergPeriodicitySelection;
            periodicityAdjustment = bloombergPeriodicityAdjustment;
            nonTradingDayFillOption = bloombergNonTradingDayFillOption;
            nonTradingDayFillMethod = bloombergNonTradingDayFillMethod;
            overrideCurrency = bloombergOverrideCurrency;
        }
        protected override void ProcessDataResponse(ref BBCOMM.Event response)
        {
            // unzip and pack messages received from BBCOMM server
            // receive one security per message and multiple messages per event
            foreach (BBCOMM.Message message in response.GetMessages())
            {
                // extract security and fields
                BBCOMM.Element security = message.GetElement(SECURITY_DATA);
                BBCOMM.Element fields = security.GetElement(FIELD_DATA);
                //
                int sequenceNumber = security.GetElementAsInt32(SEQUENCE_NUMBER);
                int nFieldNames = fieldNames.Count;
                int nObservationDates = fields.NumValues;
                //
                // the exact dimension will be known only, when the response has been received from BBCOMM server
                if (!hasDimensions)
                {
                    // define result data structure dimensions for historical data request
                    // observation date will be stored into first field for each observation date
                    result = new dynamic[securityNames.Count, nObservationDates, fieldNames.Count + 1];
                    hasDimensions = true;
                }
                //
                // loop through all observation dates
                for (int i = 0; i < nObservationDates; i++)
                {
                    // extract all field data for a single observation date
                    BBCOMM.Element observationDateFields = fields.GetValueAsElement(i);
                    //
                    // pack observation date into data structure
                    result[sequenceNumber, i, 0] = observationDateFields.GetElementAsDatetime(DATE);
                    //
                    // then, loop through all 'user-requested' fields for a given observation date
                    // and pack results data into data structure
                    for (int j = 0; j < nFieldNames; j++)
                    {
                        // pack field value into data structure if such value has been found
                        if (observationDateFields.HasElement(fieldNames[j]))
                        {
                            result[sequenceNumber, i, j + 1] = observationDateFields.GetElement(fieldNames[j]).GetValue();
                        }
                        // otherwise, pack NOT_AVAILABLE string into data structure
                        else
                        {
                            result[sequenceNumber, i, j + 1] = NOT_AVAILABLE;
                        }
                    }
                }
            }
        }
    }
}

Wednesday, December 31, 2014

Multi-Threaded Copula producer in C#

I wanted to publish one more article for the sake of celebrating the end of the year 2014. This time I am opening one possible solution for producing multi-threaded correlated random numbers with Gaussian Copula in C#. The numerical scheme used for this procedure has already been opened well enough in this blog posting. Usually, programs presented in this blog have been having (more or less) something to do with Microsoft Excel. This time, there is only C# program. However, with all the example programs presented in this blog, the user should be able to interface this program back to Excel in many different ways, if so desired.

PRODUCER-CONSUMER


In this simple example program, Producer-Consumer Pattern has been used. CopulaQueue class is nothing more, than just a wrapper for thread-safe ConcurrentQueue data structure. More specifically, the part of the program which actually creates all correlated random numbers into CopulaQueue, is the Producer. In the program, concrete CopulaEngine implementation class takes concrete RandomGenerator implementation class and correlation matrix as its input parameters. Concrete CopulaEngine implementation class then creates correlated random numbers and uses C# Event for sending created random number matrices to CopulaQueue. Another part of the program is the Consumer. Here, I have created a simple consumer class which simply prints created random numbers to console. Consumer class also uses C# Event for Dequeueing random number matrices from CopulaQueue. Needless to say, this (Consumer) is the part of our program, where we should be utilizing all correlated random numbers created by the Producer.

On design side, this program is closely following the program presented in the blog posting referred above. Again, the beef in this design is on its flexibility to allow any new implementations for Random generator and Copula model. Moreover, our example program is using C# Framework Parallel class method Parallel.ForEach, found under System.Threading.Tasks namespace. Example program creates four producers and consumers (both into List data structure) and using multi-threading (Parallel.ForEach) when processing correlated random numbers into/from CopulaQueue.


C# PROGRAM

In order to make everything as smooth as possible for the program replication, I have been writing all required classes in the same file. Just create a new console application and copy-paste the following program into a new .cs file.

using System;
using System.Diagnostics;
using System.Linq;
using System.Collections.Generic;
using System.Threading.Tasks;
using System.Collections.Concurrent;
//
namespace MultithreadingCopula
{
    // public delegate methods needed for sending and receiving matrix data
    public delegate void MatrixSender(double[,] matrix);
    public delegate double[,] MatrixReceiver();
    // ********************************************************************
    //
    // the main program 
    class Program
    {
        static void Main(string[] args)
        {
            try
            {
                // start timer
                Stopwatch timer = new Stopwatch();
                timer.Start();
                //
                // PRODUCER PART
                // create copula implementation, correlation matrix data and list of copula engines
                CopulaQueue copulaQueue = new CopulaQueue();
                double[,] correlationMatrix = createCorrelationMatrix();
                List<CopulaEngine> engines = new List<CopulaEngine>();
                int nEngines = 4;
                int nRandomArrays = 25000;
                for (int i = 0; i < nEngines; i++)
                {
                    engines.Add(new GaussianCopula(nRandomArrays, correlationMatrix, new CSharpRandom()));
                    ((GaussianCopula)engines[i]).SendMatrix += copulaQueue.Enqueue;
                }
                // process all copula engines (producers) in parallel
                Parallel.ForEach(engines, engine => engine.Process());
                //
                // CONSUMER PART
                // create list of consumers
                List<Consumer> consumers = new List<Consumer>();
                int nConsumers = nEngines;
                for (int i = 0; i < nConsumers; i++)
                {
                    consumers.Add(new Consumer(nRandomArrays, "consumer_" + (i + 1).ToString()));
                    consumers[i].ReceiveMatrix += copulaQueue.Dequeue;
                }
                // process all consumers in parallel
                Parallel.ForEach(consumers, consumer => consumer.Process());
                //
                // stop timer, print elapsed time to console
                timer.Stop();
                Console.WriteLine("{0}  {1}", "time elapsed", timer.Elapsed.TotalSeconds.ToString());
            }
            catch (AggregateException e)
            {
                Console.WriteLine(e.Message.ToString());
            }
        }
        // hard-coded data for correlation matrix
        static double[,] createCorrelationMatrix()
        {
            double[,] correlation = new double[5, 5];
            correlation[0, 0] = 1.0; correlation[0, 1] = 0.748534249620189;
            correlation[0, 2] = 0.751; correlation[0, 3] = 0.819807151586399;
            correlation[0, 4] = 0.296516443152486; correlation[1, 0] = 0.748534249620189;
            correlation[1, 1] = 1.0; correlation[1, 2] = 0.630062355599386;
            correlation[1, 3] = 0.715491543000545; correlation[1, 4] = 0.329358787086333;
            correlation[2, 0] = 0.751; correlation[2, 1] = 0.630062355599386;
            correlation[2, 2] = 1.0; correlation[2, 3] = 0.758158276137995;
            correlation[2, 4] = 0.302715778081627; correlation[3, 0] = 0.819807151586399;
            correlation[3, 1] = 0.715491543000545; correlation[3, 2] = 0.758158276137995;
            correlation[3, 3] = 1.0; correlation[3, 4] = 0.286336231914179;
            correlation[4, 0] = 0.296516443152486; correlation[4, 1] = 0.329358787086333;
            correlation[4, 2] = 0.302715778081627; correlation[4, 3] = 0.286336231914179;
            correlation[4, 4] = 1.0;
            return correlation;
        }
    }
    // ********************************************************************
    //
    // consumer class for correlated random numbers
    public class Consumer
    {
        public event MatrixReceiver ReceiveMatrix;
        public string ID;
        private int n;
        //
        public Consumer(int n, string ID) { this.n = n; this.ID = ID; }
        public void Process()
        {
            for (int i = 0; i < n; i++)
            {
                // get correlated 1xM random matrix from CopulaQueue and print it to console
                double[,] matrix = ReceiveMatrix();
                Console.WriteLine("{0}  {1}  {2}  {3}  {4}  {5}", this.ID, Math.Round(matrix[0, 0], 4), 
                    Math.Round(matrix[0, 1], 4), Math.Round(matrix[0, 2], 4), 
                    Math.Round(matrix[0, 3], 4), Math.Round(matrix[0, 4], 4));
                //
            }
        }
    }
    // ********************************************************************
    //
    // abstract base class for all uniform random generators
    public abstract class RandomGenerator
    {
        public abstract double GetUniformRandom();
    }
    // ********************************************************************
    //
    // uniform random generator implementation by using CSharp Random class
    public class CSharpRandom : RandomGenerator
    {
        private Random generator;
        //
        public CSharpRandom()
        {
            // using unique GUID hashcode as a seed
            generator = new Random(Guid.NewGuid().GetHashCode());
        }
        public override double GetUniformRandom()
        {
            // Random.NextDouble method returns a double greater than or equal to 0.0 
            // and less than 1.0. Built-in checking procedure prevents out-of-bounds random 
            // number to be re-distributed further to CopulaEngine implementation.
            double rand = 0.0;
            while (true)
            {
                rand = generator.NextDouble();
                if (rand > 0.0) break;
            }
            return rand;
        }
    }
    // ********************************************************************
    //
    // abstract base class for all specific copula implementations
    public abstract class CopulaEngine
    {
        // Uniform random generator and correlation matrix are hosted in this abstract base class.
        // Concrete classes are providing algorithm implementation for a specific copula
        // and event mechanism for sending created correlated random numbers matrix data to CopulaQueue class.
        protected RandomGenerator generator;
        protected double[,] correlation;
        //
        public CopulaEngine(double[,] correlation, RandomGenerator generator)
        {
            this.correlation = correlation;
            this.generator = generator;
        }
        public abstract void Process();
    }
    // ********************************************************************
    //
    // concrete implementation for gaussian copula
    public class GaussianCopula : CopulaEngine
    {
        public event MatrixSender SendMatrix;
        private double[,] normalRandomMatrix;
        private double[,] choleskyMatrix;
        private double[,] correlatedNormalRandomMatrix;
        private double[,] correlatedUniformRandomMatrix;
        private int rows;
        private int cols;
        //
        public GaussianCopula(int n, double[,] correlation, RandomGenerator generator)
            : base(correlation, generator)
        {
            this.cols = base.correlation.GetUpperBound(0) + 1; // M, number of variables
            this.rows = n; // N, number of 1xM matrices
        }
        public override void Process()
        {
            // NxM matrix containing of uniformly distributed correlated random variables are calculated 
            // as a sequence of specific matrix operations
            createNormalRandomMatrix();
            createCholeskyMatrix();
            correlatedNormalRandomMatrix = MatrixTools.Transpose(MatrixTools.Multiplication(choleskyMatrix, MatrixTools.Transpose(normalRandomMatrix)));
            createCorrelatedUniformRandomMatrix();
        }
        private void createNormalRandomMatrix()
        {
            // create NxM matrix consisting of normally distributed independent random variables
            normalRandomMatrix = new double[rows, cols];
            for (int i = 0; i < rows; i++)
            {
                for (int j = 0; j < cols; j++)
                {
                    normalRandomMatrix[i, j] = StatTools.NormSInv(base.generator.GetUniformRandom());
                }
            }
        }
        private void createCholeskyMatrix()
        {
            // create lower-triangular NxM cholesky decomposition matrix
            choleskyMatrix = new double[cols, cols];
            double s = 0.0;
            //
            for (int j = 1; j <= cols; j++)
            {
                s = 0.0;
                for (int k = 1; k <= (j - 1); k++)
                {
                    s += Math.Pow(choleskyMatrix[j - 1, k - 1], 2.0);
                }
                choleskyMatrix[j - 1, j - 1] = base.correlation[j - 1, j - 1] - s;
                if (choleskyMatrix[j - 1, j - 1] <= 0.0) break;
                choleskyMatrix[j - 1, j - 1] = Math.Sqrt(choleskyMatrix[j - 1, j - 1]);
                //
                for (int i = j + 1; i <= cols; i++)
                {
                    s = 0.0;
                    for (int k = 1; k <= (j - 1); k++)
                    {
                        s += (choleskyMatrix[i - 1, k - 1] * choleskyMatrix[j - 1, k - 1]);
                    }
                    choleskyMatrix[i - 1, j - 1] = (base.correlation[i - 1, j - 1] - s) / choleskyMatrix[j - 1, j - 1];
                }
            }
        }
        private void createCorrelatedUniformRandomMatrix()
        {
            // map normally distributed numbers back to uniform plane
            // process NxM matrix row by row
            for (int i = 0; i < rows; i++)
            {
                // create a new 1xM matrix for each row and perform uniform transformation
                correlatedUniformRandomMatrix = new double[1, cols];
                for (int j = 0; j < cols; j++)
                {
                    correlatedUniformRandomMatrix[0, j] = StatTools.NormSDist(correlatedNormalRandomMatrix[i, j]);
                }
                // event : send transformed 1xM matrix to CopulaQueue
                this.SendMatrix(correlatedUniformRandomMatrix);
            }
        }
    }
    // ********************************************************************
    //
    // this is technically just a wrapper class for ConcurrentQueue data structure
    public class CopulaQueue
    {
        private ConcurrentQueue<double[,]> queue;
        public CopulaQueue()
        {
            queue = new ConcurrentQueue<double[,]>();
        }
        public void Enqueue(double[,] matrix)
        {
            // insert a matrix into data structure
            queue.Enqueue(matrix);
        }
        public double[,] Dequeue()
        {
            // remove a matrix from data structure
            double[,] matrix = null;
            bool hasValue = false;
            while (!hasValue) hasValue = queue.TryDequeue(out matrix);
            return matrix;
        }
    }
    // ********************************************************************
    //
    // collection of methods for different types of statistical calculations
    public static class StatTools
    {
        // rational approximation for the inverse of standard normal cumulative distribution. 
        // the distribution has a mean of zero and a standard deviation of one.
        // source : Peter Acklam web page home.online.no/~pjacklam/notes/invnorm/
        public static double NormSInv(double x)
        {
            const double a1 = -39.6968302866538; const double a2 = 220.946098424521;
            const double a3 = -275.928510446969; const double a4 = 138.357751867269;
            const double a5 = -30.6647980661472; const double a6 = 2.50662827745924;
            //
            const double b1 = -54.4760987982241; const double b2 = 161.585836858041;
            const double b3 = -155.698979859887; const double b4 = 66.8013118877197;
            const double b5 = -13.2806815528857;
            //
            const double c1 = -7.78489400243029E-03; const double c2 = -0.322396458041136;
            const double c3 = -2.40075827716184; const double c4 = -2.54973253934373;
            const double c5 = 4.37466414146497; const double c6 = 2.93816398269878;
            //
            const double d1 = 7.78469570904146E-03; const double d2 = 0.32246712907004;
            const double d3 = 2.445134137143; const double d4 = 3.75440866190742;
            //
            const double p_low = 0.02425; const double p_high = 1.0 - p_low;
            double q = 0.0; double r = 0.0; double c = 0.0;
            //
            if ((x > 0.0) && (x < 1.0))
            {
                if (x < p_low)
                {
                    // rational approximation for lower region
                    q = Math.Sqrt(-2.0 * Math.Log(x));
                    c = (((((c1 * q + c2) * q + c3) * q + c4) * q + c5) * q + c6) / ((((d1 * q + d2) * q + d3) * q + d4) * q + 1);
                }
                if ((x >= p_low) && (x <= p_high))
                {
                    // rational approximation for middle region
                    q = x - 0.5; r = q * q;
                    c = (((((a1 * r + a2) * r + a3) * r + a4) * r + a5) * r + a6) * q / (((((b1 * r + b2) * r + b3) * r + b4) * r + b5) * r + 1);
                }
                if (x > p_high)
                {
                    // rational approximation for upper region
                    q = Math.Sqrt(-2 * Math.Log(1 - x));
                    c = -(((((c1 * q + c2) * q + c3) * q + c4) * q + c5) * q + c6) / ((((d1 * q + d2) * q + d3) * q + d4) * q + 1);
                }
            }
            else
            {
                // throw if given x value is out of bounds (less or equal to 0.0, greater or equal to 1.0) 
                throw new Exception("input parameter out of bounds");
            }
            return c;
        }
        public static double NormSDist(double x)
        {
            // returns the standard normal cumulative distribution function. 
            // the distribution has a mean of zero and a standard deviation of one.
            double sign = 1.0;
            if (x < 0) sign = -1.0;
            return 0.5 * (1.0 + sign * errorFunction(Math.Abs(x) / Math.Sqrt(2.0)));
        }
        private static double errorFunction(double x)
        {
            // uses Hohner's method for improved efficiency.
            const double a1 = 0.254829592; const double a2 = -0.284496736;
            const double a3 = 1.421413741; const double a4 = -1.453152027;
            const double a5 = 1.061405429; const double a6 = 0.3275911;
            x = Math.Abs(x); double t = 1 / (1 + a6 * x);
            return 1 - ((((((a5 * t + a4) * t) + a3) * t + a2) * t) + a1) * t * Math.Exp(-1 * x * x);
        }
    }
    // ********************************************************************
    //
    // collection of methods for different types of matrix operations
    public static class MatrixTools
    {
        public static double[,] Transpose(double[,] matrix)
        {
            // method for transposing matrix
            double[,] result = new double[matrix.GetUpperBound(1) + 1, matrix.GetUpperBound(0) + 1];
            //
            for (int i = 0; i < matrix.GetUpperBound(0) + 1; i++)
            {
                for (int j = 0; j < matrix.GetUpperBound(1) + 1; j++)
                {
                    result[j, i] = matrix[i, j];
                }
            }
            return result;
        }
        public static double[,] Multiplication(double[,] matrix1, double[,] matrix2)
        {
            // method for matrix multiplication
            double[,] result = new double[matrix1.GetUpperBound(0) + 1, matrix2.GetUpperBound(1) + 1];
            double v = 0.0;
            //
            for (int i = 0; i < matrix1.GetUpperBound(0) + 1; i++)
            {
                for (int j = 0; j < matrix2.GetUpperBound(1) + 1; j++)
                {
                    v = 0.0;
                    for (int k = 0; k < matrix1.GetUpperBound(1) + 1; k++)
                    {
                        v += matrix1[i, k] * matrix2[k, j];
                    }
                    result[i, j] = v;
                }
            }
            return result;
        }
    }
}


TEST RUN

When running the program, I get the following console printout. The printout shows, how our four consumers used in the example program are Dequeueing correlated 1x5 random matrices from CopulaQueue in parallel.

























EFFICIENCY AND CORRELATION

One million rows for 5x5 correlation matrix was processed 100 times, when testing efficiency of the producer part of the program. More specifically, four producers (CopulaEngine) are creating 1x5 matrices, consisting of five correlated uniformly distributed random numbers one by one and then sending these matrices to ConcurrentQueue wrapped in CopulaQueue class in parallel. Each of the four producers are then performing this operation 250 thousand times. Finally, the whole operation was repeated 1 hundred times and time elapsed for each repetition was saved for further analysis. The results of this experiment are reported in the table below.


As we can see in the table, multi-threading is nicely reducing processing time, but in a non-linear fashion. Next, I printed out one batch of correlated random numbers (1000000x5 matrix) into text file and compared theoretical and simulated correlation in Excel. Results are reported in the table below. We can see, that simulated correlation is in line with theoretical correlation (enough, IMHO).


READING REFERENCES


For me personally, the following two books are standing in their own class. Ben and Joseph Albahari C# in a nutshell is presenting pretty much everything one has to know (and a lot more) about C# and especially on .NET framework. Very nice thing is also, that Albahari brothers have been kindly offering a free chapter on multi-threading from this book on their website. Daniel Duffy and Andrea Germani C# for financial markets is presenting how to apply C# in financial programs. There are a lot of extremely useful examples on multi-threaded programs, especially on using multi-threading in Monte Carlo applications and the general usage of Producer-Consumer pattern, when using multi-threading.

Again, Big thanks for reading my blog and Happy and Better New Year 2015 for everybody.

-Mike