Master Thesis Worker - Automated Quality Evaluation of Developer Assistant Responses

Introduction 

Thesis work is an excellent way to get closer to Scania and build relationships for the future. Many of today's employees began their Scania career with their degree project. 

Background  

The TRATON Developer Portal provides an AI-powered Developer Assistant that helps developers find information and guidance from internal technical documentation. As its use grows, a systematic approach is needed to evaluate response quality and identify opportunities to improve both the assistant and its knowledge sources. 

Objective  

Develop and evaluate an automated approach for assessing Developer Assistant responses and generating actionable improvement recommendations. These recommendations should help distinguish documentation gaps from retrieval and agent-pipeline issues, supporting targeted improvements by the relevant teams. 

Job description
 
The thesis worker will: 

• Investigate methods for automated evaluation of AI-generated responses. 

• Analyse conversation logs and available pipeline traces. 

• Develop a prototype that identifies recurring quality issues and proposes evidence-backed actions. 

• Validate the findings with experts and assess whether selected recommendations improve response quality. 

Education/program/focus 

Indicate education, program or focus: Master’s students in Computer Science, Software Engineering, Artificial Intelligence, Data Science, Engineering Physics or a related field. 
Number of students: 1 to 2 
Start date for the thesis work: Jan-Feb 2027 
Estimated time required: 6 months 

Contact persons and supervisors

Kazi Mehedi Hasan, supervisor and fullstack developer in Traton Developer Portal team kazi.mehedi.hasan@scania.com

Schnell Melander Vilja, supervisor and industrial PhD student in Research team, vilja.schnell.melander@scania.com  

Annergren, Mariette, recruiting manager, mariette.annergren@scania.com

Application: 

Your application must include a CV, personal letter and transcript of grades  

A background check might be conducted for this position. We are conducting interviews continuously and may close the recruitment earlier than the date specified.   
   

Publication date from - to:
2026-10-26 - 2026-11-20