How big data can help identify better programmers

A new study demonstrates that data analytics can be used to evaluate the soft skills of software developers

Stop sign with the words Collaborate and Listen painted on it
Credit: flickr/rick

Contrary to popular belief outside of the tech world, software developers need to be able to do more than just write code. They also need to have the same “soft skills,” such as being able to communicate clearly and play well with others, that most people need regardless of their job. While it’s easy to evaluate a programmer’s technical chops, it’s much harder to evaluate his or her soft skills through the traditional interview process. Researchers in Singapore, however, have shown that big data can be used to identify which programmers are good at programming and have those all-important soft skills.

Those researchers, from Nanyang Technical University, conducted a 12-week study using 125 software engineering undergraduates. The idea was to collect software development behavior data of the subjects, generated through participating in Agile-driven projects. The results have been published in a new paper titled, Identifying Talented Software Engineering Students through Data-driven Skill Assessment.

Study participants self-organized into small teams  of 5 to 7 people and took part in software development projects using Scrum methodology. To track the students’ work and performance, an online tool was developed, which they called the Human-centered Agile Software Engineering (HASE) platform. Over the course of the study, the system collected data on over 10,000 activities including proposing tasks, estimating time to completion and priority, evaluating the quality of completed tasks and developer moods.

A number of metrics were calculated for each student such as:

  • Technical productivity - The amount of work a student could complete during a sprint, taking into the account the level of difficulty and the effort required.
  • Competence - The probability that a student could complete a given task with satisfactory quality by a given deadline.
  • Collaboration - The average number of people a participant collaborated with on a task.
  • Mood-stability - The change in self-reported mood from the beginning to end of each week.

Using these metrics, the authors came to a number of conclusions:

  • More competent developers could generally handle higher workloads. Not surprisingly, those who ranked high on competence also tended to have high levels of technology productivity.
  • Collaboration skills and lack of mood swings are valued. Those students who averaged the highest number of task collaborators were rated the highest by their peers, as were those whose mood was generally high and didn’t change much during the course of a sprint.
  • Technical prowess doesn’t correlate with soft skills. While almost all students in the study scored well on technical examinations, there was a wide range of performance when assessing their soft skills. The students who scored the highest on soft skills weren’t the ones with the highest technical exam scores.

What does all of this mean? The authors argued that, if software engineering programs at universities used systems like HASE for students do their work, a lot more data about the soft skills could be collected and analyzed. Issues with a lack of soft skills in students could be detected and addressed. Also, if employers had access to these data, they could make better hiring decisions about developer candidates.

Who knows? Maybe someday programmers applying for jobs, in addition to being quizzed at a white board, will also be asked for data on their collaboration ability and mood swings.

Related:
ITWorld DealPost: The best in tech deals and discounts.
Shop Tech Products at Amazon