I’m old enough that I’ve done ‘analytics’ in many venues and in many contexts. The first half of my life was in academia doing research. Within that context, the first half was essentially FORTRAN to the max with most innovation coming in the form of the hardware supporting it – mainframes to x-boxes to PC’s. The second half of my life was in essentially commercial venues and that started in a period of significant changes to computing paradigms and software: hello to the web, PHP, R, friendlier service-based paradigms, hadoop and on to todays world of cloud-based platforms, Spark, etc. Some places I’ve been in have been ‘small’ and some ‘big’ and some (academia) a weird mix of both.
I’ve been wondering lately around the pro’s and con’s of each. The big places are easy to target and make fun of but I think the usual criticisms around being too inflexible are a little too simplistic. The fact of where I am now is that it really does work to support a huge variety of ways to get work done. There’s a price though – you have to get the work done, not always the work you want to do.
There’s still a serious aspect of outsourcing activities to specialist to maintain ‘the system’ even as one has doubts about their abilities. It’s really not fair to say it’s a doubt about their ability though. These systems are designed to have support specialists who have to be agnostic about the individuals and systems they are supporting. They are not a part of your team.
While there are many tools to play with at a big place – at least my big place – there’s still always an ambient pressure to conform to a specific way of doing things which can be chafing according to your personality. Ironically, it could be my history or maybe it’s just my personality, but that friction is pretty irritating to me. For example, I’ve had my .bashrc file replaced from under me because the system is designed to support people who want to ‘do datascience’ without having to handle the day-to-day details of running a computer.
Steve's Analytic Sandbox 