Cloud Architecture training at QCon (2)

Here are the notes from today's after-lunch session of Cloud Architecture training at QCon.

We've moved on to more detailed architecture description of what is currently running under Netflix customer-facing apps. Here are the most interesting bits I noted:

1. Finally I have good names for our client and site libraries that we use in Shopzilla. Netflix calls them SAL - Service Access Library and ESL - External Service Library. I like the way they use ESL to call up local cache and that service side cache has it's own SAL and can be accessed by webapp via ESL or from service. I think our clients may be still too thick and our site libraries are too thin at the moment.

2. The main problems Netflix developers tried to solve during the switch were: the development teams interaction and the kitchen sik objects (like Movie or Customer). The first one was fixed by more service oriented architecture, with grained libraries and well defined responsibilities, for the second they used "facets pattern" which I'll describe later. Basically it's about understanding that objects can be represented differently when they are used different. It gives developers a way of working with the same objects without breaking each other.

3. Lots of good practices on logging and monitoring. Especially on how to use hadoop/hive and AWS for that. Note to self - figure out a way for us to collect usage analysis data.

4. We got a nice overview of Amazon, and Netflix tools that are used and were developed during the migration. We've heard about S3, SQS, some third party tools like GeoIp and keystore /HSM. There are some new open source tools going to be releases soon by Netflix - watch this space: https://github.com/Netflix

5. We also heard about how they deal with security, making sure that only right people have access to the instances, but also limiting the ways services can cal each other - only services that are part of certain security group can call a service, easy way to find out who calls who.