Showing posts with label Pandas. Show all posts
Showing posts with label Pandas. Show all posts

Sunday, July 14, 2019

QuantLib-Python: flexible construction scheme for piecewise yield term structures

I consider QuantLib to be a fundamental pricing library, which can effectively handle valuations for pretty much any given type of security. If there is no existing implementation for an instrument available, one can create a new implementation for it. What then makes the use of QuantLib library sometimes difficult? It's the amount of work to be done, before anything will happen. Outside of that promised functionality to value security, one has to take full responsibility of all involved janitor work. The code is (usually always) containing endless sections for different variable definitions and creation of different types of helper objects. Even creating realistic pricing scheme for a simple interest rate swap seems to require an army of different variables and objects. A lot of cooking anyway, before the beef will be served.

In this post, one possible scheme for flexible construction of QuantLib piecewise yield term structures will be presented. The program and all involved files can be downloaded from my GitHub repository.

Assume we would like to construct piecewise yield term structure for EUR and USD. Assume also, that we have the following market data for EUR and USD currencies in a specific CSV file.

Ticker,Value
USD.DEPOSIT.1D,0.02359
USD.DEPOSIT.1W,0.0237475
USD.DEPOSIT.1M,0.02325
USD.DEPOSIT.2M,0.0232475
USD.DEPOSIT.3M,0.0230338
USD.FUTURE.2M,97.92
USD.FUTURE.5M,98.005
USD.FUTURE.8M,98.185
USD.FUTURE.11M,98.27
USD.FUTURE.14M,98.33
USD.SWAP.2Y,0.01879
USD.SWAP.3Y,0.01835
USD.SWAP.5Y,0.01862
USD.SWAP.7Y,0.0194
USD.SWAP.10Y,0.02065
USD.SWAP.15Y,0.02204
USD.SWAP.30Y,0.02306
EUR.DEPOSIT.1D,-0.00366
EUR.DEPOSIT.1W,-0.00399
EUR.DEPOSIT.1M,-0.00393
EUR.DEPOSIT.3M,-0.00363
EUR.DEPOSIT.6M,-0.00342
EUR.FUTURE.5M,100.48
EUR.FUTURE.8M,100.505
EUR.FUTURE.11M,100.505
EUR.FUTURE.14M,100.495
EUR.FUTURE.17M,100.47
EUR.SWAP.1Y,-0.0038
EUR.SWAP.2Y,-0.0039
EUR.SWAP.5Y,-0.0019
EUR.SWAP.7Y,-0.0002
EUR.SWAP.10Y,0.0024
EUR.SWAP.15Y,0.0056
EUR.SWAP.30Y,0.008

In essence, we have key-value pairs in this CSV file, where the key is ticker (instrument, such as deposit, future or swap) and the value is rate (or price for a futures contract). Now, all instruments in that file are following some specific market conventions. All these conventions are then stored in a specific JSON file. The content of this file can be easily understood by using some available JSON editor.






















In essence, we are actually storing all required "constant" parameters for all QuantLib instruments we would like to use for constructing piecewise yield curves, into this file. At the moment, there are required conventions available for constructing EUR and USD curves.

Next, we have builder class PiecewiseCurveBuilder for assembling QuantLib piecewise yield term structures, as shown below. In the first stage, we store all conventions and market data into this builder class in its constructor. After this, builder is ready for constructing curves. As client is requesting specific curve by using Build method, the class will then create all bootstrap helpers based on a given market data (for requested currency) and instrument conventions (for instruments in requested currency).

# create piecewise yield term structure
class PiecewiseCurveBuilder(object):
    
    # in constructor, we store all possible instrument conventions and market data
    def __init__(self, settlementDate, conventions, marketData):        
        self.helpers = [] # list containing bootstrap helpers
        self.settlementDate = settlementDate
        self.conventions = conventions
        self.market = marketData
    
    # for a given currency, first assemble bootstrap helpers, 
    # then construct yield term structure handle
    def Build(self, currency, enableExtrapolation = True):

        # clear all existing bootstrap helpers from list
        self.helpers.clear()
        # filter out correct market data set for a given currency
        data = self.market.loc[self.market['Ticker'].str.contains(currency), :]
        
        # loop through market data set
        for i in range(data.shape[0]):            
            # extract ticker and value
            ticker = data.iloc[i]['Ticker']
            value = data.iloc[i]['Value'] 
            
            # add deposit rate helper
            # ticker prototype: 'CCY.DEPOSIT.3M'
            if('DEPOSIT' in ticker):
                # extract correct instrument convention
                convention = self.conventions[currency]['DEPOSIT']
                rate = value
                period = ql.Period(ticker.split('.')[2])
                # extract parameters from instrument convention
                fixingDays = convention['FIXINGDAYS']
                calendar = Convert.to_calendar(convention['CALENDAR'])
                businessDayConvention = Convert.to_businessDayConvention(convention['BUSINESSDAYCONVENTION'])
                endOfMonth = convention['ENDOFMONTH']
                dayCounter = Convert.to_dayCounter(convention['DAYCOUNTER'])
                # create and append deposit helper into helper list
                self.helpers.append(ql.DepositRateHelper(rate, period, fixingDays, 
                    calendar, businessDayConvention, endOfMonth, dayCounter))
        
            # add futures rate helper
            # ticker prototype: 'CCY.FUTURE.10M'
            # note: third ticker field ('10M') is defining starting date
            # for future to be 10 months after defined settlement date
            if('FUTURE' in ticker):
                # extract correct instrument convention
                convention = self.conventions[currency]['FUTURE']
                price = value
                iborStartDate = ql.IMM.nextDate(self.settlementDate + ql.Period(ticker.split('.')[2]))
                # extract parameters from instrument convention
                lengthInMonths = convention['LENGTHINMONTHS']
                calendar = Convert.to_calendar(convention['CALENDAR'])
                businessDayConvention = Convert.to_businessDayConvention(convention['BUSINESSDAYCONVENTION']) 
                endOfMonth = convention['ENDOFMONTH']
                dayCounter = Convert.to_dayCounter(convention['DAYCOUNTER'])
                # create and append futures helper into helper list
                self.helpers.append(ql.FuturesRateHelper(price, iborStartDate, lengthInMonths,
                    calendar, businessDayConvention, endOfMonth, dayCounter))                
            
            # add swap rate helper
            # ticker prototype: 'CCY.SWAP.2Y'
            if('SWAP' in ticker):
                # extract correct instrument convention
                convention = self.conventions[currency]['SWAP']
                rate = value
                periodLength = ql.Period(ticker.split('.')[2])
                # extract parameters from instrument convention
                fixedCalendar = Convert.to_calendar(convention['FIXEDCALENDAR'])
                fixedFrequency = Convert.to_frequency(convention['FIXEDFREQUENCY']) 
                fixedConvention = Convert.to_businessDayConvention(convention['FIXEDCONVENTION'])
                fixedDayCount = Convert.to_dayCounter(convention['FIXEDDAYCOUNTER'])
                floatIndex = Convert.to_iborIndex(convention['FLOATINDEX']) 
                # create and append swap helper into helper list
                self.helpers.append(ql.SwapRateHelper(rate, periodLength, fixedCalendar,
                    fixedFrequency, fixedConvention, fixedDayCount, floatIndex))
        
        # extract day counter for curve from configurations
        dayCounter = Convert.to_dayCounter(self.conventions[currency]['CONFIGURATIONS']['DAYCOUNTER'])
        # construct yield term structure handle
        yieldTermStructure = ql.PiecewiseLinearZero(self.settlementDate, self.helpers, dayCounter)
        if(enableExtrapolation == True): yieldTermStructure.enableExtrapolation()
        return ql.RelinkableYieldTermStructureHandle(yieldTermStructure)

The final component in this scheme is Convert class, which performs conversions from string presentation to specific QuantLib data types. This class is heavily used in builder class, where convention string information is transformed into correct QuantLib data types.

# utility class for different QuantLib type conversions 
class Convert:
    
    # convert date string ('yyyy-mm-dd') to QuantLib Date object
    def to_date(s):
        monthDictionary = {
            '01': ql.January, '02': ql.February, '03': ql.March,
            '04': ql.April, '05': ql.May, '06': ql.June,
            '07': ql.July, '08': ql.August, '09': ql.September,
            '10': ql.October, '11': ql.November, '12': ql.December
        }
        s = s.split('-')
        return ql.Date(int(s[2]), monthDictionary[s[1]], int(s[0]))
    
    # convert string to QuantLib businessdayconvention enumerator
    def to_businessDayConvention(s):
        if (s.upper() == 'FOLLOWING'): return ql.Following
        if (s.upper() == 'MODIFIEDFOLLOWING'): return ql.ModifiedFollowing
        if (s.upper() == 'PRECEDING'): return ql.Preceding
        if (s.upper() == 'MODIFIEDPRECEDING'): return ql.ModifiedPreceding
        if (s.upper() == 'UNADJUSTED'): return ql.Unadjusted
        
    # convert string to QuantLib calendar object
    def to_calendar(s):
        if (s.upper() == 'TARGET'): return ql.TARGET()
        if (s.upper() == 'UNITEDSTATES'): return ql.UnitedStates()
        if (s.upper() == 'UNITEDKINGDOM'): return ql.UnitedKingdom()
        # TODO: add new calendar here
        
    # convert string to QuantLib swap type enumerator
    def to_swapType(s):
        if (s.upper() == 'PAYER'): return ql.VanillaSwap.Payer
        if (s.upper() == 'RECEIVER'): return ql.VanillaSwap.Receiver
        
    # convert string to QuantLib frequency enumerator
    def to_frequency(s):
        if (s.upper() == 'DAILY'): return ql.Daily
        if (s.upper() == 'WEEKLY'): return ql.Weekly
        if (s.upper() == 'MONTHLY'): return ql.Monthly
        if (s.upper() == 'QUARTERLY'): return ql.Quarterly
        if (s.upper() == 'SEMIANNUAL'): return ql.Semiannual
        if (s.upper() == 'ANNUAL'): return ql.Annual

    # convert string to QuantLib date generation rule enumerator
    def to_dateGenerationRule(s):
        if (s.upper() == 'BACKWARD'): return ql.DateGeneration.Backward
        if (s.upper() == 'FORWARD'): return ql.DateGeneration.Forward
        # TODO: add new date generation rule here

    # convert string to QuantLib day counter object
    def to_dayCounter(s):
        if (s.upper() == 'ACTUAL360'): return ql.Actual360()
        if (s.upper() == 'ACTUAL365FIXED'): return ql.Actual365Fixed()
        if (s.upper() == 'ACTUALACTUAL'): return ql.ActualActual()
        if (s.upper() == 'ACTUAL365NOLEAP'): return ql.Actual365NoLeap()
        if (s.upper() == 'BUSINESS252'): return ql.Business252()
        if (s.upper() == 'ONEDAYCOUNTER'): return ql.OneDayCounter()
        if (s.upper() == 'SIMPLEDAYCOUNTER'): return ql.SimpleDayCounter()
        if (s.upper() == 'THIRTY360'): return ql.Thirty360()

    # convert string (ex.'USD.3M') to QuantLib ibor index object
    def to_iborIndex(s):
        s = s.split('.')
        if(s[0].upper() == 'USD'): return ql.USDLibor(ql.Period(s[1]))
        if(s[0].upper() == 'EUR'): return ql.Euribor(ql.Period(s[1]))        

Finally, let us take a look, how easily we can actually construct QuantLib piecewise yield term structures for our two currencies. After this point, these constructed curves can then be used as arguments for pricing engines.

# create instrument conventions and market data
rootDirectory = sys.argv[1] # command line argument: '/home/mikejuniperhill/QuantLib/'
evaluationDate = Convert.to_date(datetime.today().strftime('%Y-%m-%d'))
ql.Settings.instance().evaluationDate = evaluationDate
conventions = Configurations(rootDirectory + 'conventions.json')
marketData = pd.read_csv(rootDirectory + 'marketdata.csv')

# initialize builder, store all conventions and market data
builder = PiecewiseCurveBuilder(evaluationDate, conventions, marketData)
currencies = sys.argv[2] # command line argument: 'USD,EUR'
currencies = currencies.split(',')

# construct curves based on instrument conventions, given market data and currencies
for currency in currencies:    
    curve = builder.Build(currency)
    # print discount factors semiannually up to 30 years
    times = np.linspace(0.0, 30.0, 61)
    df = [round(curve.discount(t), 4) for t in times]
    print('discount factors for', currency)
    print(df)

Address to root directory (containing market data and instrument conventions files) and requested currencies are given as command line arguments. Conventions object and market data (pandas data frame) are being created and fed to curve builder object in its constructor. Finally, curves are being requested for each currency.

It can be seen, that we can actually create piecewise yield term structures for any currency (as long as market data and conventions are available) with a very few lines of code by using this kind of construction scheme. In essence, all the complexity involved is still there, but we have effectively moved all "janitor work" into specific classes (builder, conversions) and files (conventions, market data).

Configuration for a new currency should be straightforward: add instruments into market data file (ticker-value-pairs). Then, add new section for this new currency to conventions file (including configuration sub-sections for all instruments in this new currency). Also, some minor additions might be needed in conversion class.

Program execution is shown below.








Thanks for reading my blog.
-Mike

Wednesday, July 3, 2019

Python: using JSON file for increasing program configurability

As the experience tells us, wrong decisions in program design and life will usually bite back hard. In order to avoid the most obvious traps leading into horrific maintenance problems, we should always design our programs to be free of any hard-coded parameters. By using configuration scheme presented in this post, flexible programs, which can use any desired set of input configurations, can be created. This means we can (as an example) execute a specific program (for valuing batch of transactions) several times, but using different set of configurations (different set of market data) for each execution. All example files can be downloaded from my GitHub page.

In this very simple example, Python program will just print a set of market and fixings data from CSV files, based on a given set of configurations. In order to keep this example program short and sweet, our JSON configuration file has only two configurations: directory addresses for market and fixings data CSV files, as follows.

{
  "MARKETDATA":"/home/mikejuniperhill/Market.csv",
  "FIXINGSDATA":"/home/mikejuniperhill/Fixings.csv"
}

Directory address of this configuration file will be given as a command line argument for the program. Based on this given configuration, program will then read configured data from files to be used in program. 
















Example program is shown below. In the first stage, program will create configurations object. Technically, this object is just a wrapper for dictionary data structure. Any specific configuration can be accessed by using Python version of index operator overloading. After this, program reads data from configured CSV files into DataFrame objects and prints their contents to terminal.

import json
import sys
import pandas

# class for hosting configurations
class Configurations:
    inner = {}
    # read JSON configuration file to dictionary
    def __init__(self, filePathName):
        self.inner = json.load(open(filePathName))
        self.inner = {k.upper(): v for k, v in self.inner.items()}
    # return value for a given configuration key
    # 'overload' indexing operator
    def __getitem__(self, key):
        return self.inner[key.upper()]

# configuration file string is command line argument
configurationsFilePathName = sys.argv[1]

# create configurations object
config = Configurations(configurationsFilePathName)

# create market data based on configuration
market = pandas.read_csv(config['MarketData'])
print('EUR swap curve:')
print(market.head())

# create fixings data based on configuration
fixings = pandas.read_csv(config['FixingsData'])
print('6M Euribor fixings:')
print(fixings.head())

Handy tool for constructing and testing syntactic correctness of any JSON file can be found in here. Finally, thanks for reading.
-Mike

Tuesday, January 8, 2019

Python: Market Scenario Files Generator for Third-party Analytics Software

Third-party analytics software usually requires specific set of market data for performing its calculations. In this post, I am publishing one of my utility Python programs for creating different types of stress scenario markets, based on given base market and set of prepared XML configurations. The complete program can be found in my GitHub repository.

Configurations


The following screenshot shows configurations for this program. SourceFilePath attribute captures the source market data CSV file and TargetFolderPath captures the folder, into which all market scenario files will be created. Finally, ScenarioConfigurationsPath captures the folder, which contains all XML scenario configuration files. This configuration XML file should be stored in a chosen directory.

<Configurations>
    <!-- attributes for scenario generator settings -->
    <ScenarioConfigurationsPath>\\Temp\ScenarioConfigurations\</ScenarioConfigurationsPath>
    <SourceFilePath>\\Temp\baseMarket.csv</SourceFilePath>
    <TargetFolderPath>\\Temp\Scenarios\</TargetFolderPath>
</Configurations>

Market data


The following screenshot shows given base market data. Due to brevity reasons, only EUR swap curve has been used here as an example. All market data points are defined here as key-value pairs (ticker, value).

IR.EUR-EURIBOR.CASH-1BD.MID,-0.00365
IR.EUR-EURIBOR.CASH-1W.MID,-0.00373
IR.EUR-EURIBOR.CASH-1M.MID,-0.00363
IR.EUR-EURIBOR.CASH-2M.MID,-0.00336
IR.EUR-EURIBOR.CASH-3M.MID,-0.00309
IR.EUR-EURIBOR.CASH-6M.MID,-0.00237
IR.EUR-EURIBOR-6M.FRA-1M-7M.MID,-0.00231
IR.EUR-EURIBOR-6M.FRA-2M-8M.MID,-0.00227
IR.EUR-EURIBOR-6M.FRA-3M-9M.MID,-0.002255
IR.EUR-EURIBOR-6M.FRA-4M-10M.MID,-0.00218
IR.EUR-EURIBOR-6M.FRA-5M-11M.MID,-0.00214
IR.EUR-EURIBOR-6M.FRA-6M-12M.MID,-0.002075
IR.EUR-EURIBOR-6M.FRA-7M-13M.MID,-0.00198
IR.EUR-EURIBOR-6M.FRA-8M-14M.MID,-0.00186
IR.EUR-EURIBOR-6M.FRA-9M-15M.MID,-0.001775
IR.EUR-EURIBOR-6M.FRA-10M-16M.MID,-0.00169
IR.EUR-EURIBOR-6M.FRA-11M-17M.MID,-0.00159
IR.EUR-EURIBOR-6M.FRA-12M-18M.MID,-0.00141
IR.EUR-EURIBOR-6M.SWAP-2Y.MID,-0.001603
IR.EUR-EURIBOR-6M.SWAP-3Y.MID,-0.000505
IR.EUR-EURIBOR-6M.SWAP-4Y.MID,0.00067
IR.EUR-EURIBOR-6M.SWAP-5Y.MID,0.00199
IR.EUR-EURIBOR-6M.SWAP-6Y.MID,0.003315
IR.EUR-EURIBOR-6M.SWAP-7Y.MID,0.0046
IR.EUR-EURIBOR-6M.SWAP-8Y.MID,0.00584
IR.EUR-EURIBOR-6M.SWAP-9Y.MID,0.00698
IR.EUR-EURIBOR-6M.SWAP-10Y.MID,0.00798
IR.EUR-EURIBOR-6M.SWAP-11Y.MID,0.00895
IR.EUR-EURIBOR-6M.SWAP-12Y.MID,0.009775
IR.EUR-EURIBOR-6M.SWAP-15Y.MID,0.01165
IR.EUR-EURIBOR-6M.SWAP-20Y.MID,0.013245
IR.EUR-EURIBOR-6M.SWAP-25Y.MID,0.013725
IR.EUR-EURIBOR-6M.SWAP-30Y.MID,0.01378

We can clearly see, that the system used for constructing market data tickers leads to scheme, in which every market data point will have one and only one unique ticker. This will then guarantee, that we can drill down and stress individual market data points with regex expressions, if so desired. This data should be copied into CSV file (directory has been defined in previous configuration file).

Scenario configurations


The following screenshot shows XML configurations for one market scenario. One such scenario can have several different scenario items (Say, stress these rates up, stress those rates down, apply these changes to all FX rates against EUR and set hard-coded values for all CDS curves). From these configurations, ID and description are self-explainable. Attribute regExpression captures all regex expressions (scenario items), which will be searched from risk factor tickers. As soon as regex match is found, the program will use corresponding operationType attribute to identify desired stress operation (addition, multiplication or hard-coded value). Finally, the amount of change which will be applied in risk factor value is defined within stressValue attribute. This XML configuration should be stored (directory has been defined in program configuration file).

<!-- operation types : 0 = ADDITION, 1 = MULTIPLICATION, 2 = HARD-CODED VALUE -->
<Scenario>
  <ID>CURVE.STRESS</ID>
  <description>custom stress scenario for EUR swap curve</description>
  <regExpression>^IR.EUR-EURIBOR.CASH,^IR.EUR-EURIBOR-6M.FRA,^IR.EUR-EURIBOR-6M.SWAP</regExpression>
  <operationType>1,0,2</operationType>
  <stressValue>1.25,0.015,0.05</stressValue>
</Scenario>

Finally, the following screenshot shows resulting market data, when all configured scenario items have been applied. This is the content of output CSV file, created by this Python program.

IR.EUR-EURIBOR.CASH-1BD.MID,-0.0045625
IR.EUR-EURIBOR.CASH-1W.MID,-0.0046625
IR.EUR-EURIBOR.CASH-1M.MID,-0.0045375
IR.EUR-EURIBOR.CASH-2M.MID,-0.004200000000000001
IR.EUR-EURIBOR.CASH-3M.MID,-0.0038624999999999996
IR.EUR-EURIBOR.CASH-6M.MID,-0.0029625000000000003
IR.EUR-EURIBOR-6M.FRA-1M-7M.MID,0.01269
IR.EUR-EURIBOR-6M.FRA-2M-8M.MID,0.01273
IR.EUR-EURIBOR-6M.FRA-3M-9M.MID,0.012745
IR.EUR-EURIBOR-6M.FRA-4M-10M.MID,0.01282
IR.EUR-EURIBOR-6M.FRA-5M-11M.MID,0.01286
IR.EUR-EURIBOR-6M.FRA-6M-12M.MID,0.012924999999999999
IR.EUR-EURIBOR-6M.FRA-7M-13M.MID,0.01302
IR.EUR-EURIBOR-6M.FRA-8M-14M.MID,0.013139999999999999
IR.EUR-EURIBOR-6M.FRA-9M-15M.MID,0.013224999999999999
IR.EUR-EURIBOR-6M.FRA-10M-16M.MID,0.013309999999999999
IR.EUR-EURIBOR-6M.FRA-11M-17M.MID,0.01341
IR.EUR-EURIBOR-6M.FRA-12M-18M.MID,0.01359
IR.EUR-EURIBOR-6M.SWAP-2Y.MID,0.05
IR.EUR-EURIBOR-6M.SWAP-3Y.MID,0.05
IR.EUR-EURIBOR-6M.SWAP-4Y.MID,0.05
IR.EUR-EURIBOR-6M.SWAP-5Y.MID,0.05
IR.EUR-EURIBOR-6M.SWAP-6Y.MID,0.05
IR.EUR-EURIBOR-6M.SWAP-7Y.MID,0.05
IR.EUR-EURIBOR-6M.SWAP-8Y.MID,0.05
IR.EUR-EURIBOR-6M.SWAP-9Y.MID,0.05
IR.EUR-EURIBOR-6M.SWAP-10Y.MID,0.05
IR.EUR-EURIBOR-6M.SWAP-11Y.MID,0.05
IR.EUR-EURIBOR-6M.SWAP-12Y.MID,0.05
IR.EUR-EURIBOR-6M.SWAP-15Y.MID,0.05
IR.EUR-EURIBOR-6M.SWAP-20Y.MID,0.05
IR.EUR-EURIBOR-6M.SWAP-25Y.MID,0.05
IR.EUR-EURIBOR-6M.SWAP-30Y.MID,0.05

Handy way to create and test regex expressions is to use any online tool available. As an example, the first scenario item (^IR.EUR-EURIBOR.CASH) has been applied to a given base market data. The last screenshot below shows all regex matches.






















Have a great start for the year 2019 and thanks a lot again for reading my blog.
-Mike