Data Engineer Consultant for ASR (Automatic Speech Recognition) at Farm Radio International

Scope of Work
The consultant will be responsible for the following tasks:

• Data Management: Collect, clean, and preprocess audio and textual data to train and evaluate ASR models.
• Data Pipeline Development: Design and implement scalable data pipelines to automate the extraction, transformation, and loading (ETL) of speech data.
• Model Training: Work closely with the data science team to prepare data for training, validating, and testing ASR models.
• Performance Monitoring: Monitor and analyze the performance of ASR models, including accuracy, latency, and error rates. Identify and address any data-related issues.
• Integration: Collaborate with software developers to integrate ASR models into web applications, ensuring seamless functionality and optimal user experience.
• Documentation: Create and maintain documentation related to data workflows, model performance, and integration processes.
• Collaboration: Work with cross-functional teams including product manager, UX/UI designers, and data scientists to align data engineering efforts with overall product goals.

Deliverables
Data Collection and Management:

• Data collection scripts and tools for gathering speech and text data.
• Clean and labeled datasets for training and evaluation purposes.
• Data storage solutions with appropriate indexing and retrieval mechanisms.

Data Pipeline Documentation:

• Design documents for data pipelines, including flow diagrams and architecture.
• Source code and configurations for ETL processes.

Model Preparation:

• Scripts and processes for preparing and augmenting data for ASR model training.
• Documentation on dataset splits (training, validation, test).

Model Performance Reports:

• Performance metrics reports for ASR models, including accuracy, precision, recall, and F1 scores.
• Analysis and troubleshooting reports on any data-related issues impacting model performance.

Integration Documentation:

• Integration guides and API documentation for incorporating ASR models into web applications.
• Code samples and best practices for interacting with ASR services from web clients.

User Feedback Analysis:

• Reports and insights from user feedback related to ASR accuracy and usability.
• Recommendations for improvements based on user feedback and performance data.

Codebase and Version Control:

• Version-controlled code repository for data processing scripts and pipeline implementations.
• Regular updates and commits to the codebase reflecting ongoing improvements and bug fixes.

Collaboration Reports:

• Documentation of cross-functional team meetings and decisions.
• Progress reports on collaborative projects involving ASR integration.

Testing and Quality Assurance:

• Test cases and results for ensuring data quality and model accuracy.
• Documentation of QA processes and issues encountered during testing.

Training and Support Materials:

• Training materials and documentation for team members on data handling, model integration, and troubleshooting.

Duration and Timeline:

• The consultancy will commence in October 2024 and conclude in August 2025 with a maximum level of effect (LoE) of 80 days.
• Specific deliverable deadlines will be agreed upon at the start of the consultancy.

Reporting:

• The consultant will report to the Project Manager Nathaniel Ofori and provide regular updates on progress.
• Weekly meetings will be held to discuss challenges, milestones, and next steps.