Job description
We are looking for a highly skilled and experienced Data Engineer to join our team as a (part-time or full-time) freelance contractor to help set up an effective pipeline for characterizing complex and highly diverse component designs, generate photonic measurement data, and create an automated data quality control system.
Key tasks
• Measurement Automation:
• Maintain and improve a Python-based software solution used for automating measurement systems.
• Integrate new instruments and communication protocols (e.g., GPIB, USB, RS232, Ethernet) into the existing system.
• Troubleshoot and resolve issues in real-time data acquisition and instrument communication.
• Data Analysis Software:
o Develop new data analysis algorithms and improve the performance of existing ones.
o Ensure the system can handle large datasets efficiently, adding new features requested by the scientific team.
o Work closely with users to implement and test new data analysis methods.
• Web API Development & Azure Dat
Profile / Requirements :
Profile
• Proficiency in Python, with experience in backend development, system automation, and data analysis.
• Experience developing and maintaining software that automates measurements and interfaces with instruments using protocols like GPIB, RS232, USB, and Ethernet.
• Experience developing or maintaining Python-based data analysis software, including handling large datasets and optimizing performance for scientific or experimental data processing.
• Strong experience in web-based API development with a focus on performance, scalability, and integrity.
• Experience working with Azure cloud databases (e.g., Azure SQL Database, Cosmos DB), optimizing queries for performance, and ensuring fast and secure data access and storage.
• Proficient in using Git for version control and collaboration within development teams.
• Setting up data pipelines.
• Automated data quality control (e.g. using AI) and warning system.
• Experience in using AI for data mining is a plus