ITQuants blog

Fusion Invest: the right way to implement a cache for portfolio columns using the C# toolkit

Nov 30

Written by:
11/30/2014 9:37 PM

I've already implemented several times user columns at the clients of Misys. Each time, taking a look at the developments already done by the client or specific deliverables made by Misys, I encounter the same problems of performance, which are:

- in most cases, no memory cache is implemented, calculations are done on the fly, with, last but not least, direct access to the database, sometimes hidden by the complexity of the Sophis API,

- if a memory cache is implemented, the refresh is not necessary very well optimized. In most cases, no SEC listener is developped and the refresh is done according to the version given by CSRPortfolioColumn::GetRefreshVersion.

- calculations of aggregations are done on CSRPortfolioColumn::GetPositionCell (or CSMPortfolioColumn.GetPositionCell in C#) instead of being implemented in the CSRGlobalFunctions::StartPortfolioAddition or EndPortfolioCalculations, with no indication on the level on which the aggregation should start. It means that for each refresh needed, the algorithm implemented will parse the whole portfolio hierarchy each time. If all folios are loaded, the parsing can freeze the whole GUI.

The goal of this post is then to give the rules for implementing user columns as it should.

On the previous and current version of Fusion/Sophis, the display of the cells content are optimized as follow:

- the graphic control does not own any specific data, but refers to the one stored in some memory cache (known as CSMgrFolio). Access to this data is done by calling callbacks like CSRPortfolioColumn::GetPositionCell,

- when drawing the portfolio columns, it calculates which lines should be displayed, and for each line, displays the columns one after the others.

Thus, for user columns, the best is to have one cache that stores the data needed, it means one cache for all loaded lines and for all columns displayed at screen. Calculations of aggregated lines should be done on CSR(M)GlobalFunctions::StartPortfolioAddition or at least in EndPortfolioCalculations. In order to optimize the parsing of the folios, on very big folios, the best is to parse all needed lines, and to calculate all displayed columns for each line, instead of having one cache per column, and parsing for each column the whole folio hierarchy.

In C#, I've seen some clients who implemented the memory cache using string key like "folio=folioId position=positionId instrument=instrumentId. Such a key has a poor performance compared to the one with a specific structure and an a HashCode calculated as follow:

public class ITQPositionKey

{

private KeyType _type;

private int _folioId;

private int _positionId;

private int _instrumentId;

public enum KeyType

{

Position,

Folio,

Underlying

}

public override int GetHashCode()

{

const int p = 17, n = 31;

int result = p;

result = Type.GetHashCode() + result * n;

result = FolioId.GetHashCode() + result * n;

result = PositionId.GetHashCode() + result * n;

result = InstrumentId.GetHashCode() + result * n;

return result;

}

...

Some purists would say that even using another (n,p) algorithm, it could increase the performance. I won't discuss this point since compared to a string key, it is in any case faster.

Concerning the point on how to override StartPortfolioAddition or EndPortfolioCalculations, I will refer to the specific article already posted there: http://www.it-quants.com/Blogs/tabid/83/EntryId/59/Sophis-Value-Risque-how-to-override-the-global-functions-in-C-when-using-previous-versions-6-3.aspx. By the way, on last versions of FusionInvest, the CSMGlobalFunctions support the Register method and does not need some workaround as specified in this article.

Concerning the point on how to get the list of portfolio columns displayed at screen, the best is to consult the following article: http://www.it-quants.com/Blogs/tabid/83/EntryId/61/Fusion-Invest-how-to-know-which-portfolio-columns-are-displayed.aspx.

Performance can be increased by storing the last key used and the corresponding value. The unitary key corresponds to the position (see above), it means a line. Since the system displays each column one after the others, instead of looking for some line in the cache, we just need to store all values of the line as follow:

public ITQPositionField GetValue(ITQPositionKey key, int colId)

{

....

if (key == _previousKey)

{

if (_previousValue != null && _colIndexes.ContainsKey(colId))

return _previousValue.Fields[_colIndexes[colId]];

else

throw new Exception("Key+value not found, folioId=" + key.FolioId.ToString() +

", positionId=" + key.PositionId.ToString() +

", instrumentId=" + key.InstrumentId.ToString());

}

...

On the implementation proposed below, I've created an interface IITQPositionCalculation, which permits to implement the way on how the data is retrieved from database or API, and the way on how it is aggregated (for instance, on Last, Allotment.. columns, it makes no sense to compute some aggregation). The registration of these algorithms is done using the singleton ITQPositionCalculationManager object. The data for all columns displayed at screen is stored in a memory cache named ITQColumnCache. The aggregated lines are stored in this cache too. The refresh of the cache is done on EndPortfolioCalculations. If some external event should change the content of this cache, specific listener should be implemented. That's not the case in the provided code.

In order to test the implementation, several columns were developed, in order to test the way the aggregation is done. They are grouped under the ITQuants configuration group. There is no added value in the columns, since it corresponds to the Sophis native columns Result, Balance, Realized, UnRealized and Treasury columns. It permits to test the coherency of the resuls.

Finally, some screenshot: