2012 Agile Software Development for Healthcare summit materials

Note: Since I became news editor for InfoQ (you can view my posts here) I do not post to kreskanotes very often. I do have some posts that I wrote with infoQ in mind that didn't make it to the page, so I will be putting them here so the effort is not completely lost. They will be tagged with infoQ label.

The Agile Software Development for Healthcare summit, organised this Autumn in Berlin by Pharma IQ, hopes to repeat the success of the Software Design for Medical Devices events that took place in the United States in the past few years. The aim of the conference is to “create a valuable platform for knowledge and experience sharing with an integrated mix of theoretical and case study driven presentations”. The organisers prepared a lot of research and downloadable materials that you might find interesting, even if you don’t plan on participating in the conference.
The materials which you can access on the summit’s web page contain:

• the Agile Software Info Pack, a document which “(..) summarizes the key issues related to new agile development in healthcare, changes in clinical evaluations for medical devices and tips on the agile techniques for a project and address the regulatory challenges of developing a medical device.”
• “What Agile Software Development Learned from Deming”, an article prepared for Pharma IQ by the contributors from W. Edwards Deming Institute. It explains how the roots of modern agile approach can be seen in Dr. W. Edwards Deming’s management and quality work of the 1980's.
• “Conception to Completion: Why Agile Development for Healthcare”, a two part podcast series in which Brian Shoemaker of  Shoebar Associates and Nancy Van Schooenderwoert, a Consultant at Lean Agile Partners “(...) discuss the main reasons for changing the approach to software development, give best practice tips on the agile techniques for a project and address the regulatory challenges of developing a medical device with respect to software development pitfalls encountered.“
• Agile Software Development in a Medical Device Context, an audio interview with Michael Meissner, the VP Software R&D and Program Manager at Omnyx LLC, in which he talks about possible downstream effects of Agile adoption. You can find more details about this interview in this press release.

Software Development Tips you can learn from Xing 4.0, a different interview, which allows the listeners to pick up useful tips and lessons from Traian Kaiser, Director of Agile Project Management and PMO at Xing.

Qcon 2012, Where does Big Data meet Big Database

I've listened to the "Where does Big Data meet Big Database" byBenjamin Stopford talk at London's Qcon. The talk was mostly covering the differences between the "traditional" databases and the new technologies that were introduced in last decade. It provided a helpful insights into what questions one should answer when deciding on which technology to use, and unsurprisingly "which one is sexier" was not one of those questions. Here are my notes from the talk:

1. The situation in data sources domain is much different that 10 years ago. The scale changes, we have new big sources of data that people and companies are interested in, we have a range of new technologies, products and ideas that developers can leverage when architecting their applications.

2. Map Reduce is gaining popularity since 2004. It is simple, pragmatic, it solved a problem that didn't have a solution before, it was novel and not limited by the tradition that stands behind older ideas. It introduced a split between so called "hacker culture" and "enterprise culture". Around 2009 it was criticised as "not really novel", operating on a wrong level (physical not logical level), incompatible with tooling, lacking referential integrity and elegance.

3. The new approach can be characterised as bottom-up approach to accessing very large data sets, that allows us to keep the data close to the code (close to where it matters) and to keep it in it's natural form.

4. Relational approach ("old" approach) - solid, consistent and safe data, limiting developers by it's language.

5. As early as 2001 some people closely connected with "traditional" solutions realised that there are problems with existing DB - that they are good tools, but they force developers to "speak different language", they need to evolve and allow more flexibility. The NoSQL movement came about because developers wanted a simple storage that works on diverse hardware, that scales, and can be talked to in developer's language of choice. "One size fits all" is not true anymore.

6. When choosing the tool you need to consider following solutions:  In-Memory DB, Shared Everything, Shared Nothing, Column Oriented Storage,

7.  In-Memory DB - fast, but doesn't scale well and you can loose your data very easily. This will probably gain more popularity as the distributed-in-memory solution evolves.

8. Shared Everything - all the data is located on a shared disc space, all the client nodes have access to full data set, every node can handle every request. This is pretty slow, and every cache needs to sit above whole database, which means the bigger the data set the less effecting your caching solution is.

9. Shared Nothing - each node has the access to one shard of the full data set. This means you need to find the node that can handle your request - worst case scenarion you'll need to iterate through all your nodes.

10. Column Oriented Storage - data organised in columns not rows, laid contiguously on disc. Very good compression.  Indexing becomes less important, sigle record pull is quite expensive, but bult read/write is really faster.

11. Additional factors: discs - older DBs were optimised for sequential access over magnetic drives not random access over SSD, growing speed of our networks. Mind that there are relational databases that leverage latest technologies - they can achieve very good results in benchmarks.

12. When comparing and deciding on solution don't only think about the size of your data set, it's potential to grow or even only about abilities of each tool. Think about what You need/want. Do you mind SQL? Is your data changing a  lot constantly, is the data isolated? do you want to/can you keep your data in the natural state? which solution you can afford?

13. It is good that we do have a range of tools that provide different solutions for different problems, the trick is to know what is your problem, so you can pick the right solution. 

Qcon 2012: Technology is your office

I've attended the "Technology is your office" talk by Horia Dragomir this morning. Most of the session covered good practices to be followed when working with distributed teams of developers. It could be summed up with talk with each other, be smart and create a culture people can share. What I found interesting was the list of tools Horia recommended for distributed teams, so here they are:

"Management" (as in management of time and workload, not resources):

• Skype
• Google apps
• Google docs
• Basecamp
• Redmine
• Pivotaltracker
• JIRA
• Rally

Development

• Mercurial 
•  Github
• Yuuguu
• Cloud9 IDE
• everytimezone.com
• TeamViewer

Design

• Mocking Bird
• Balsamiq
• Axxure
• photoshopetiquette.com 
• Layer Vault 

Also bonus points for the eastern-european accent joke at the beginning of the talk and neat idea to come up with a catch phrase that sums up your team culture, so it can be propagated more easily. 

Qcon 2012, Keynote - Resilient Response in Complex Systems

I was a bit late to the Friday's keynote at London's QCon (thanks, Southern trains) , but I managed to take some notes from John Allspaw's "Resilient Response in Complex Systems" talks. Here they are:

1. There are number of things that go wrong when the teams are faced with a failure of a complex system.  There's refusal to make decisions caused by either lack of authority (people want to make a decision but they are not sure if they can), lack of information (people can't make a decision, they can only guess) or bureaucracy and politics (people that are able to make a decision can't make it fast or without some approvals). There is "heroism" - individuals who walk away from the team to focus on what they think is the solution for the problem. If they success they send a wrong message to the team and the company - that the issues are solved by individuals, if they fail they abandoned the team when facing a disaster. There are also distractions - you need to be able to cut down the distractions to minimum  when dealing with a disaster. This means irrelevant emails/links/social events but can also mean isolating business owners from the team if they only add distractions when "panicing" over the outage.

2.  SRK Framework: http://en.wikipedia.org/wiki/Ecological_interface_design#The_Skills.2C_Rules.2C_Knowledge_.28SRK.29_framework

3. A good example on how to deal with a disaster can be found in so called High Reliability Organisations (HRO - for example companies which work can affect human life or health, places like air traffic control or hospital). They are very complex, they have to trade off between efficiency and thoroughness, they usually engineer-driven.

4. "Managing the Unexpected: Resilient Performance in an Age of Uncertainty" by Karl E. Weick 

5. The ways HRO deal with disaster: the teams are close to each other, they share the tools and the information, there is overlap in skills and knowledge, the team members can be moved from one team to another. Over-communication is the norm. There is safe environment in which teammates can point out errors and mistakes - reporting of errors and mistakes is awarded.  There is high level of "unease" - everybody is used to idea that disaster will happen, and they are prepared for it. People have high level interpersonal skills. There are publicly available detailed records of previous incidents. Patterns of authority are dynamically changing to meet the demands - people directly dealing with the problem have the power to fix it.

6. Drill - practice troubleshooting under pressure, be comfortable with your tools,try to come up wit new ways your system can break, and practice the ways of dealing with those. The are actions that can be taken immediately when a disatser happens, make sure those are fast and automatic for your team.

7. The law of stretched systems

8. Near misses - communicate them and widely distribute them. They happen more often than actual distaters so you get much more volume of incidents to improve on. They are a reminder of the risks.

9. Spend as much time understanding why your team succeeded as on why the team failed. When faced with choice wether to analyse your success of your failure choose to analyse both. Think what and why things go right.

10. Learn from other fields, train for outages, learn from mistakes (and avoid politics and bureaucracy), learn from successes as well as failures.  

Qcon 2012 ,230 Iterations later

I've wandered off to the Agile track at London's QCon today, and listened to "230 Iterations later" talk by Suki Bains and Kris Lander. Here are my notes from the track:

The main idea was to use example of the team both speakers worked with in past 4 years to show how being agile, focused on good process and delivery can lead to "delivery zombies" - teams that only focus on delivering stories without asking how? and why?

1. They did the right thing at the beginning and it worked well. The team was piloting agile approach in the company, it took on new project, started off with brave decision of building up a new platform as a backend for the new site. Managed to successfully deliver but also to build a "perfect" agile environment, strong foundation for future projects, and a team that felt they work towards common goal.

2. Reflecting now on this time speakers wondered what they mean "good" when they say the team was in a good place back than. Does "good team" mean a team that produces code that provides required features? Is it a feeling that team members have?

3. Thanks to the success the team build up reputation, good relationship with business owners, delivery of features became easy. Shipping became a mesure of success. Some early signs of future problems started showing up: the goals the team had in front of it felt small, the technical debt started building up, the "how?" question being answered the process felt too easy/boring, and the team started making mistakes - slipping. Task weren't always picked up in priority order ("This is first in the queue but it's a lot of front end,  I can't work on it"). Sounds scary when I think about my team... :S

4. After 160 iterations the team became a "delivery zombie". Focused only on shipment of the features, not interested in innovation. The office became quiet. Mind that the code was still well maintained, there was high level of automation in the platform. The external test were covering most of functionality (even though they were a "nightmare to manage"), the cost of adding new features were kept down. The code is still considered to be an asset.

5. 176 iterations is a new project is introduced. It's in line with what the team was doing before and they feel confident they can deliver again. The situation changes when the business owners change. The team looses 1-to-1 relationship with product owners, new structure brings on new complexity, and the team can't adapt quickly enough. The feedback loop between development and business is broken and the team loses it's identity.

6. 190 iterations in there are conflicts in the team, there is no common purpose, and people are not working together anymore. The team realises that delivery of the features can't be it's measure of success anymore - they are still delivering, but they can see that's not enough. They struggle to define new goals and values.

7. At the end the projects is still shopped on time, and there is a positive feedback both from business and users. But the team during retrospective doesn't feel successful. The takeaway: vision is important, needs to be reinforced often and on different levels. The team needs to share common values and goals, and needs to understand (and ask) why? as well as how? Keep on asking what it means for your team to be "good", "successful" and "right".