Sunday, April 26, 2015

Handling survey data : XML or JSON?

For a few weeks I have been pondering how best to handle 'survey data' for an organization that collects a lot of survey data and needs to make it available to people for a variety of 'business intelligence' needs.

The key needs are:
- must be in a format that is easy for the contributor of the survey data
- must allow for the same surveys to be repeated over time e.g. quarterly survey, for the same survey respondents e.g. Company ABC completes survey a number of times over a period of time
- must allow for survey results to be easily compared and contrasted e.g. same contributor, different survey submissions ( this quarter versus last quarter) and different contributors for the same time period e.g. Company ABC versus Company XYZ
- allow for the content of surveys to change over time e.g. questions can be added, deleted, changed
- allow people to access the data as surveys e.g. all data for a given survey's instance (all data for this quarter) or accessing the data for a given question over time e.g. answers to question 4 this quarter versus last quarter
- allow easy mapping of survey data to broader business concepts e.g question 2's answer is 'annual revenue' for a given respondent 

*whew* I think that's about it.... easy, huh?

much of the  research stuff I read suggested:
- XML is a good way of representing a survey and collecting survey data
- translate the XML data into a relational data model view of the survey and store in a SQL database

... sounds great but nothing a read really convinced me there could be a simple, generic approach that would work for all situations.

Keeping the data in an XML format, to allow data to be accessed in its survey form seems to hold some promise for addressing some of the requirements, at least partially, but thability to easily query the XML data seemed clumsy, at best... enter JSON

JSON seems to provide similar advantages for receiving survey data in a structured way and, with the advent of modern document databases, also the ability to easily query the data in its survey form

more to follow in this series... :)

Virgin Business Class to Brisbane

Today I am flying Virgin Business Class to Brisbane :)  I must say its a great experience so far Here's breaky ... Bircher muesli with macadamia nuts and yoghurt, fresh fruit, sparkling mineral water and a coffee to come *yum*

I'm in seat 1C, sitting in front of Dawn Fraser, trying to work out if I should say 'hello' :)

I'm catching up in Bris-Vegas and Gold Coast with some good friends I have not seen in years... looking forward to it :)

I gave up a chance to sit next to Dawn

Today I am flying to Brisbane, to catch up with some friends I have not seen in a while.  When I was checking in for the flight the lady at check in asked me if I preferred row 1 or 2, window or aisle.  I had pre booked row 1 on the aisle so I stayed with that. Once on the flight, seated and ready for departure, on walked Dawn Fraser and she quickly found her way to row 2.  There's a vacant seat next to her... that could have been mine *sigh*

Oh well, I'm sitting in front of Dawn instead,,,, maybe the title of this blog should be.... "Glenn beats dawn to finish line in Brisbane!" :D