The Science behind operating a successful Scientific Software team

During the pandemic, I ran both scientific and information technology teams within a high throughput start up environment. As the battle against COVID-19 progressed there became a greater need within the laboratory for synergy between scientists and software. I was tasked with building a support team that managed the laboratory software (LIMS - laboratory information management system) in the context of the scientific process.

OBSERVATIONS

GAP analysis revealed what was already becoming evident, the scientific workforce operated without a technical understanding of the software. Software training for scientific staff was absent with the exception of three scientists who had attended a LIMS training course. This course however did not describe how the software architecture fitted together. These steps were the cornerstone of the automated track system responsible for managing everything from file transfer to robotics (see figure 1).

Figure 1: Workflow for the process to be successful

Not only was the overlap in knowledge base inadequate to realise management's goal of a more cohesive work force, the associated standard operating procedures (SOPs) detailing scientific processes omitted key information regarding the LIMS. While the SOPs referring to LIMS existed without the context of the science, the same pattern emerges, again we see each discipline existing in a vacuum unaware of the other.

SHORT TERM GOAL SETTING

The most important goal, I felt, was to ensure the user felt comfortable with the LIMS and use this as a platform from which to grow the user's confidence. Ultimately culminating in the user being able to question the system’s logic and raise issues or suggest improvements where necessary.

I began by changing the content of the SOPs, namely the scientific SOPs were updated to include LIMS steps. I then created useful e-learning materials which were used to train staff remotely, quickly and in large numbers. Levels of competence were scrutinised with short assessments that users could implement as evidence to demonstrate basic levels of competence within the software.

Incident management process flows were produced to demonstrate the correct way to escalate incidents by raising tickets using a website service. Knowledge base articles were created for common incidents. Making these incident management protocols also served a second purpose of explicitly categorising the different types incident. When communicating problems in senior meetings it became easier to perform root cause analysis and answer questions such as; Was this user error or lack of training? or Was this a development request in the user requirement specification?

TEAM BUILDING AND TRAINING

Having developed clear stepping stones for existing staff to navigate the software it was time to build a hybrid team combining scientific and information technology expertise to correctly support software queries and trouble shooting. The hiring process included an interview, and a competence evaluation designed to test the candidate’s ability to find: anomalies and formatting errors, null data, data missingness, and identifiable patterns within an excel spreadsheet without time to prepare. While performance in the task had to be of a certain level, it also allowed insight into a candidate’s ability to work under pressure, arguably a more important skill than being an ‘excel wizard’.

Once assembled, the software support team were trained on the most commonly observed tickets in a test environment, such as managing duplicate barcodes. The “fake incidents” within test were assessed and ensured that the team were prepared to go on to shift. There was a requirement for the team to perform an end-to-end smoke test of the system to ensure that they understood exactly how the system worked giving special attention to the most challenging areas commonly requiring actions to be performed in the correct order. Once assembled and trained, my team ran a 24/7 service for the laboratory.

REFINE AND OPTIMISE

To begin with, the level of urgency assigned to raised tickets was subjective. But over time the scientific staff learnt the typical severity level for different incidents and tickets were raised with more appropriate priorities. 24/7 operations come with many challenges one of the largest being the nature of shift work, required to ensure 24/7 coverage. Allowing enough time for handovers between shifts ensured that incident information was correctly communicated. This was incredibly valuable in the overlap period between shift changes, typically allotting a 2-hour periods to talk through the backlog of incidents.

EMPOWERMENT AND PRIVILEGES

As the scientific team were given more training, the SOPs were revised and system developments halted, the team grew more confident with the software. The overall result of this was a decline in the number of incidents being raised. Users were given more privileges and responsibilities while corresponding SOPs were written helping to further lower the number of incidents with a subsequent increase in user functionality.

RINSE AND REPEAT

Fortunately, software support is somewhat like a cookie cutting process, once you have established a good ethos with all those involved; scientists, major incident managers and support teams, a trust can grow that the incidents will be resolved and users will feel empowered.

It became easy for this mentality to spread to new systems implemented. The lessons learned above resulted in successful deployment of the ReflX application with appropriate service design documentation. In a perfect world, a software support team is not required when the users are well educated in the software, the correct level of empowerment is granted in a carefully cultivated ethos of trust. However, superusers and software support teams are highly recommended in the laboratory spaces and their value should not be underestimated.