After working full time for one year, one of the stumbling blocks for (good) analysts producing results is tools.

There is such a huge emphasis on what tools you should be using, rather than how you should use it! Big Data is all the rage and interest right now, and perhaps that might be what is to blame.

##A Shifting Focus? The industry may be moving in the right direction. With the introduction of MOOCs like Udacity, Coursera, and EdX, vigilant and proactive computer programmers or statisticians can increasingly improve and widen their knowledge.

This is important.

Machine learning takes many shapes and forms, and it isimportant that we realise that generic machine learning fails (quick google will show you many, many examples). But as computer programmers improve their statistics, and understand the importance of assumptions, what p-values actually mean, they will be able to craft their own algorithms with increasing precision through their increased statistical knowledge.

Likewise, as statistician improve their programming, they will be able to make use of these algorithms and craft faster and more robust algorithms in this age of Big Data. Since I stand within this camp, contrary to what the Actuarial education believes, or what my degree has provided me, the fact that regex has been ignored from the curriculum (and programming in general), I truly believe we can not rely on degrees to provide solid, real world applications to theoretical statistics.

##Criticisms of my University Degree?

It pains me to say that in hindsight I do not believe my degree (or actually looking at the MOOCs) provide the real-world training/expectation. Nor do I really have an alternative viewpoint/philosophy.

In fact the most useful information I gathered from my university degree was not in statistics or actuarial science, but instead within applied mathematics; on the calculation of flops and the big O notation for the analysis of algorithms. (Unfortunately throughout the whole course only two weeks was spend on this since this is applied mathematics, not computer science).

###Spreadsheets are not the answer

Spreadsheets feel like the bane of Big Data. *.xls format only fits \(2^{16} \approx 65,000\) rows (for the SAS export functionality) and *xlsx format doesn’t actually fare much better from a practical standpoint with \(2^{20} \approx 1,000,000\).

Yet within Australia, within my Actuarial degree, there was a portion of my degree dedicated to “Spreadsheet Standards” (to be fair it was actually useful, but if you were terrible in the first place, you’ll just end up with a bruised dead), not to mention peer supported VBA classes. These classes existed due to the industries desire for VBA skills. Now again in hindsight, I still don’t understand why VBA was seen as a necessary skill.

Though for the Actuarial industry, we can understand why. They don’t actively deal with big data. Spreadsheets are “good enough” (at least in Australia). In fact, if actuarial analysts needed data, tidy data will be provided to them from designated database administrators!

###It’s My Fault

I would conclude that my criticisms of my University degree fall on me and me alone, for not pursuing more “practical” computing courses.

But then again, how would I have known I would arrive in my current banking role?