Thomas Sun

NavInfo MIFG POI data conversion tool which is converting NavInfo MIFG format POI data to HERE Platform supported Protocol Buffers format data:

• Participated in stand-ups, task planning, issue or bug solving
• Building required infrastructure for optimal extraction, transformation and loading of data from various data sources using AWS and SQL and Scala, Spark technologies like Spark RDD operations , Dataset, Lazy Evaluation, Transformations, Spark SQL, Spark Collections
• Used SQL and Spark transformations, different collections and RDD operations for processing SQLite file for each province and fetch and assemble required fields for each POI data
• Used programming languages like Java and Scala in Spark Framework to collect all fields required for each POI and aggregate hundreds of POIs into Google Protocol Buffer format block of file
• Designed with team leader and colleagues predefined mapping table between NavInfo format and HERE format for data conversion
• Leverage AWS storage services for data storage to be delivered to HERE teams for places search
• Developed, designed and implemented Jenkins pipeline service to automate data pre-process, data conversion and data upload for storage

Developer of Automation Testing Platform which is in charge of the corresponding HERE services in China market, for example, routing service, geocoding service, POI search service, reverse geocoding service:

• Fully in charge of project maintenance, bug fixing, features adding
• Designed test scripts and procedures with team colleagues to maximise team efficiency and catch all system flaws
• Reviewed requirements, specifications and other project documentation to develop automated test scripts
• Worked well in Agile environment to effectively plan and carry out different tests on HERE services at staging environment and production environment
• Leveraged version control systems to manage source code, track changes and facilitate collaboration among team members
• Updated existing testing system to resolve errors and adapt to supporting multiple different tests running simultaneously
• Created technical documents to detail code and design of new added features
• Developed high-quality software solutions using Java,Python,Ruby programming languages and Shell Scripting adhering to code standards
• Newly implemented test cases pass or failure will not impact the existing test cases
• Different tests can be run at the same time simultaneously without interfering with each other and improving the overall system reliability

Data Conversion Jenkins Pipeline which contains several Jenkins jobs to accomplish the whole conversion process for NavInfo MIFG conversion

• Fast learn fundamentally basic DevOps concepts such as automation of software development lifecycle, collaboration between teams, continuous integration and continuous delivery to increase efficiency and fast delivery, and minimize resource waste and human errors
• Fast learn pipeline tools like GitLab functions, Jenkins services, and microservices deployment and AWS services
• Managed cloud-based infrastructure, optimizing costs and ensuring high availability of services.
• Reduced time-to-delivery, automating build, test, and deployment pipelines using CI/CD tools for data conversion
• Downloaded NavInfo MIF_G data and uncompress and sort out files by the corresponding province by Shell programming
• Leveraged ogr2ogr utility to input MIF_G data file into SQLite file database by each province in Shell programming
• Launched multiple AWS EMR clusters for data conversion in parallel to speedup the whole process in Shell programming
• Downloaded each file piece from all provinces and packed them back to archived file and uploaded to AWS S3 storage
• Run China legal Testing process to check POI conversion and content moderation in Shell script

Website Automation which contains a variety of test cases to ensure the serving websites complies with the laws and regulations locally

• Interacted with team leader to gain strong understanding of project and testing objectives
• Worked within Agile frameworks to consistently meet project timelines and deliverable objectives
• Collected and reviewed website to analyze system capabilities and development requirements
• Ensured functions like HERE repositories in China, registered users information in China, different services running in China all work
• Ensured copyright information in this website before login and after login can be displayed normally
• Ensured that the necessary fields required for registering a new user, email support when users run into problems, the existing users can invite new users, users can modify password all can work normally
• This automation has been integrated with China Legal Testing System for HERE service call

Testing tool for NavInfo MIFG POI data which checks if the data complies with laws locally and HERE specifications:

• Collaborate with different teams to form data mapping relationships between NavInfo MIFG format and HERE platform required Protocol Buffers
• Used Python language fetching different data fields of each POI by traversing different MIFG files based on mapping table
• Used Programming languages like Java and Scala to retrieve data for each POI
• Used Java and Scala to accomplish data comparison process and output all the differences for each POI
• Reported output to team leader for further analysis

SOW5 project which intended to measure the performance metrics and functions for non-performance

• Worked with team leader to prototype technical ideas and propose solutions
• Dived deep into HERE website documentation, for HERE services and API and SDK
• Found different APIs and SDKs for HERE different services, such as topology layer, navigation layer, places layer, cartography layer in HERE Map Content catalog
• Worked with team leader to figure out relationships between HERE Standard Map and HERE High-Definition Map, and found correlation
• Worked with team leader in different cases of matching standard map road link with high definition map road link
• Programmed in different HERE SDK such as Spark SDK, Flink SDK in HERE platform pipeline to measure different storage services like versioned layer and volatile layer speed and cost and data persistency
• Programmed in Spark SDK in HERE platform to find if there is any unmatched between HD map road topology segment and Standard Definition road topology segment
• Programmed in different SDK such as Flink, Spark or Python against the same kind of layer like version layer or volatile layer to measure different SDK performances

Cennevi(NavInfo wholly-owned subsidiary) Real Time Transportation Project which required Cennavi XML format transformation data imported into HERE traffic layer

• Developed software solutions by studying information needs and collaborating with teammates
• Developed rich client solutions for applications with Scala programming skills
• Dived deep into HERE documentation for Flink SDK API operations with volatile layer
• Decoded data by HERE schema and inspected data fields of traffic, such as traffic light, road topology segment, segment length, weather conditions
• Extracted specified data fields from XML data of Cennavi provider and HERE protocol buffer data from HERE volatile layer, and found missed information
• Output logging information to HERE version layer for further issue tracking