AI camera to reduce user effort and operation time in user reward program
A driver incentive program allows drivers to display company stickers on their vehicles in exchange for enhanced rewards and priority in order allocation. Participants must submit monthly photos showing their vehicle with the visible company sticker to continue receiving program benefits.
This initiative has successfully increased brand recognition while simultaneously improving driver engagement.
Our team has optimized the verification process to minimize submission errors and reduce the time required for drivers to complete their monthly checks.
↓ 15%
Reduce in rejection rate
↓ 43hrs
Reduce in monthly agent hours
Lalamove is an on-demand delivery platform that connects 13 million users with 1.2 million driver partners in 11 markets across the world. Lalamove driver-partner app is a tool for drivers to receive and manage delivery requests from customers.
My role
- Designed the user experience and user interface
- Structured and defined the problem to solve after testing
Opportunity
Despite the oppourtunity,
The opportunity is
How can we reduce submission rejections, helping drivers capture photos more quickly while minimizing resubmissions and decreasing the number of follow-up calls local agents need to make?
Goal
The product manager set the goal and success metrics for this feature. We want to target:
- Reduce rejected submissions
- Reduce operation time to review and call rejected drivers
- Reduce the number of rejected photos due to “License plate was not fully visible”
Steps
Understand Problem
The monthly verification requires drivers to take photos showing the company sticker from 3 different angles, with the license plate visible in each image to confirm vehicle matches registration records.
The app simply instructs what drivers need to take while taking a photo in texts.
*During initial sticker installation at the Lalamove office, the operations team guides drivers through the photo taking process, demonstrating the proper technique for monthly submissions.
Data analysis
PM and the operation team found out that main 3 reasons of the rejections are accounted for nearly 90% of the rejections.
- License plate was not fully visible
- Photo quality is low
- Sticker is not fully visible
Define Scope
Focus on “License plate was not fully visible”
We decided to focus on the license plate display due to the high volume. We previosuly developed
Potential impact
The data shows that 40% of rejections are caused by not showing the license plate. A new feature should be designed to target this directly. While the theoretical maximum impact is a 40% reduction, we are setting our official target at a confident 15% reduction.
We have 50,000 sticker submission every month and the rejection rate is around 10%.
Operation team has to call drivers whose sticker submissions are rejected and it is estimated to take 3 minutes to call
50,000 drivers * 10% rejection rate * targeted 20% reduction in rejection * 3 minutes per driver / 60 (Mins -> hours)
→ estimated 37.5 hours reduction in operation team workload
Hypothesis about the feature value
In order to make sure if we solve the problem, we set hypothesis:
If we add object recognition to the license plate in the monthly photo check, then we will reduce invalid photo submissions and eliminate approximately 50 hours of operation team workload.
Design MVP
I designed MVP for the solution.
Overall flow
As a driver with sticker, I want to have instructions on how to scan the license plate and stickerso that I understand what I need to do next while completing the tasks
To reduce technical effort and maintain a consistent user experience, we modeled this feature on a similar existing one. Animated guides walk drivers through each step, and they can reference a sample photo at any time. To avoid confusion, the instructions only mention the license plate, as our data shows drivers rarely struggle with sticker visibility.
As a driver with sticker, I want to verify my photo against the example, so that I can prevent it from being rejected and having to do it again.
I propose replacing the pre-capture document guide with a post-capture comparison step. This allows drivers to directly compare their photo against a sample image, which is more effective than asking them to recall earlier instructions.
Cognitive load theory
As a driver with sticker, I want a manual camera option, so that I'm not blocked if the AI detection has an issue.
To prevent users from getting stuck, a fallback mechanism is required in case of AI camera failure.
Assess 1st Testing
We launched the feature in 3 cities to gauge the impact.
Rejection rate with breakdown by reasons in 3 cities
The results were disappointing. While the overall rejection rate improved only marginally, rejections due to “Photo quality is low” alarmingly increased.
Define Problem to Solve
Focusing on solving the increase in “Photo quality is low”
In order to investigate the root cause of the increase holistically, I structured the issue by 4 segments: process, displays, systems, and people.
How can we validate or invalidate that the new camera design causes an issue to photo quality?
While the tech team was investigating the potential system issues, I was able to analyze the potential issues with usability.
Research goal
The goal was to:test a hypothesis that the new camera causes usability issues, which leads drivers to submit photos that don’t show sticker and license plate clearly.
The “Photo quality is low” reason is frequently selected by the operation team because it serves as a catch-all for various issues, such as blurriness, poor framing, or improper lighting.
Methodologies and key questions
We observed and asked 3 drivers in person to tested both the old and new camera flows in the app to see if there is any change in behaviors or issues.
Participant criterias:
- Drivers who failed the photo check in the testing month with the reason "photo quality is low" or "Sticker is not fully visible"
- Drivers who passed the photo check in the previous month
Key questions to answer are:
- How do drivers take photos of vehicles with the old camera and new camera?
- How different is photo-taking between the old camera and the new camera?
- What are the pain points of drivers when they take photos with the new camera?
The testing invalidated the hypothesis that the new design would cause usability issues. All participants were able to take photos without much struggle, and all resulting images met our requirements.
Other findings from the testing
01
Drivers generally ignore the captions in the old camera, appearing to rely on existing knowledge of the workflow.
02
User preference on the cameras was divided. While some drivers found the instructions in the new camera helpful, others preferred the simplicity of the old camera, which lacks a detection function.
03
No one uses zoom in and out function of the camera, they move themselves to get the right angle. Only one takes horizontally
What are the potential system-related causes of reduced image quality?
After the tech team investigations, we found out a few things related to systems that cause photo quality to be lower.
01
On older devices, the video frame can result in lower-quality photos due to hardware limitations.
02
The internal system automatically blurries
After the app captures a photo, it is sent to an internal system for temporary storage before being forwarded to a driver management portal for agents to review.
03
An aspect ratio incompatibility between the taken photos and the portal causes cropping.
To provide drivers with a larger capture area, the camera frame was set to full screen. However, this creates an issue on devices with non-standard aspect ratios (i.e., ratios other than 16:9 or 4:3). This problem is particularly prominent on Android devices, which feature a wide variety of screen dimensions.
Design 1st Iteration
In response to the system issues we identified, we implemented the following solutions.
Compress the photo compression by 80% to ensure the photo quality from the video frame and fit 5 MB file size limit.
To meet the portal's aspect ratio requirement, we changed to the 9:16 ratio from the original camera.
Hypothesis about the enhancement
If we implement the compression strategy and change the camera frame ratio, then we will see the reduction in the rejection reasons, “Photo quality is low”.
Assess 2nd Testing
This time, we launched A/B testing of the feature to a small batch of drivers in 2 smaller cities for the whole month and launched to a small batch of drivers in 1 big city for 2 weeks.
A/B testing: Rejection rate with breakdown by reasons
AI Camera improved rejection rates by 10% over the original flow, with fewer rejections due to “License plate was not fully visible”.
Comparison to previous month: Rejection rate with breakdown by reasons
Volume and rejection rate by camera type
This confirms AI camera's effectiveness in reducing rejection rates. However, over 30% of drivers utilized the fallback option when not using AI. It’s proven that AI camera can lower the rejection rate.
Define Problem to Solve
Priority is to reduce the usage of fall back option among drivers
The testing results proved AI camera's superior effectiveness, which led us to encourage its use over the fallback option.
70% of drivers opted for the fallback camera instead of continuing with the AI camera when given a choice.
After 5 seconds of failed license plate detection, a dialog appears.
Most drivers instantly select the primary button, suspecting they choose the most visually prominent option.
The 5-second timeout does not provide drivers sufficient time to to detect a license plate.
With a median completion time was 15 seconds; the 5-second timeout is insufficient. We believe the short timeout prevents successful AI camera use, forcing drivers to resort to the fallback option.
Design 2nd Iteration
Extend timeout to 10 seconds and reverse button positions to reduce fallback camera usage
We extended the insufficient 5-second timeout to 10 seconds as a balanced compromise that addresses stakeholder concerns about user frustration from longer waits.
We are also exploring whether users choose the fallback camera simply because it appears as the most visually prominent option in the dialog.
Hypothesis about the enhancement
If we increase the timeout to 10 seconds and reverse the button positions, then we expect fewer drivers will choose the fallback option.
Assess 3rd Testing
We launched a consistent month-long A/B test across all three cities, reflecting our increased confidence compared to our previous testing.
A/B testing: Rejection rate with breakdown by reasons
AI Camera outperformed the original flow by 14%, showing further improvement from previous testing while maintaining fewer rejections due to 'License plate was not fully visible.”
Comparison with previous tests: Rejection rate with breakdown by reasons
Rejection rate remained similar before the change, but the composition shifted as license plate issues decreased drastically while low photo quality problems increased.
The shift might be related to our larger driver rollout compared to the second test and warrants monitoring.
Volume and rejection rate by camera type
AI camera adoption increased while demonstrating continued effectiveness over fallback cameras. Only 19% of drivers used the fallback camera at least once, representing a 37% decrease from the previous version.
Define the final testing goal
Launch A/B testing in the rest of the operating cities to gauge the effectiveness of the feature
Different markets show different driver behaviors, so we needed to verify the feature's effectiveness across all cities. During this process, stakeholders suggested several small guide improvements, which we incorporated alongside the A/B testing.
Design 3rd Iteration
Updated the pointer guide to mention the sticker, ensuring drivers understand they need to capture both the license plate and sticker in their photos
The original guide only referenced the license plate, prompting stakeholder concerns about not specifically mentioning the sticker, which is an essential component of sticker retention requirements. However, data shows that sticker visibility has not been an issue in practice.
Added a preliminary guide that appears before drivers begin the process, providing clear instructions on proper photo-taking techniques
The local operations team expressed concerns that insufficient detailed guidance could confuse drivers. In response, we added a comprehensive guide featuring an instructional image with specific tips, similar to the original design.
Modified the dialog text that appears when the AI camera hasn't detected a license plate within 10 seconds to assist drivers in capturing a proper image with helpful tips
We leveraged the 10-second detection failure point as an opportunity to provide drivers with helpful tips, encouraging them to try again rather than immediately resorting to the fallback option
Assess 4th Testing
We implemented A/B testing across all cities, excluding the 3 original launch cities, to verify that the AI camera improves the rejection rate consistently regardless of location.
A/B testing: Rejection rate with breakdown by reasons
AI camera outperformed the original camera by 19%, demonstrating consistent effectiveness across all global markets.
Final result
We fully deployed the feature to all drivers across all cities for a one-month period.
Rejection rate with breakdown by reasons
Compared to the month before launch, we achieved precisely our target of a 15% improvement in overall rejection rates, while simultaneously reducing license plate visibility issues by a substantial 76%.
The number of drivers with stickers grew by 7,000 while the testings. We still assumed that agents take 3 mins per driver to follow up on the rejected drivers.
57,000 drivers with stickers * (8.6-10.1%) rejection rate * 3 mins per driver / 60
→ 43 agent hours reduced monthly
Next steps
We can explore the implementation of object detection technology for vehicle stickers.
Since both license plates and stickers must be clearly visible, implementing sticker-specific object detection could significantly reduce rejections categorized as 'Sticker is not fully visible,' complementing our existing success with license plate detection.
We can research more on how agents choose rejection reasons.
Now that 'Photo quality is low' has emerged as the predominant rejection reason, we need to investigate how agents are using this category. It appears to be functioning as a catch-all for various issues. We should conduct further analysis to break down the specific violations within this category, allowing us to identify and address the underlying problems more effectively.
AI camera to reduce user effort and operation time in user reward program
A driver incentive program allows drivers to display company stickers on their vehicles in exchange for enhanced rewards and priority in order allocation. Participants must submit monthly photos showing their vehicle with the visible company sticker to continue receiving program benefits.
This initiative has successfully increased brand recognition while simultaneously improving driver engagement.
Our team has optimized the verification process to minimize submission errors and reduce the time required for drivers to complete their monthly checks.
↓ 15%
Reduce in rejection rate
↓ 43hrs
Reduce in monthly agent hours
Lalamove is an on-demand delivery platform that connects 13 million users with 1.2 million driver partners in 11 markets across the world. Lalamove driver-partner app is a tool for drivers to receive and manage delivery requests from customers.
My role
- Designed the user experience and user interface
- Structured and defined the problem to solve after testing
Opportunity
Drivers aim to complete tasks efficiently, including the monthly sticker verification photo required for rewards. When rushing, they often miss critical requirements, resulting in rejected submissions. The most common issue is failing to include the license plate in the photo. When rejected, drivers must either resubmit or forfeit their program benefits.
We' explored object recognition technology to guide drivers in capturing photos that clearly show both the company sticker and license plate on their vehicles, streamlining the verification process and reducing rejected submissions.
This also decreases operational workload for local agents, who had to contact drivers with rejected submissions.
The opportunity is
How can we reduce submission rejections, helping drivers capture photos more quickly while minimizing resubmissions and decreasing the number of follow-up calls local agents need to make?
Goal
The product manager set the goal and success metrics for this feature. We want to target:
- Reduce rejected submissions
- Reduce operation time to review and call rejected drivers
- Reduce the number of rejected photos due to “License plate was not fully visible”
Steps
Understand Problem
The monthly verification requires drivers to take photos showing the company sticker from 3 different angles, with the license plate visible in each image to confirm vehicle matches registration records.
The app simply instructs what drivers need to take while taking a photo in texts.
*During initial sticker installation at the Lalamove office, the operations team guides drivers through the photo taking process, demonstrating the proper technique for monthly submissions.
Data analysis
PM and the operation team found out that main 3 reasons of the rejections are accounted for nearly 90% of the rejections.
- License plate was not fully visible
- Photo quality is low
- Sticker is not fully visible
Define Scope
Focus on “License plate was not fully visible”
We decided to focus on the license plate display due to the high volume. We previosuly developed
Potential impact
The data shows that 40% of rejections are caused by not showing the license plate. A new feature should be designed to target this directly. While the theoretical maximum impact is a 40% reduction, we are setting our official target at a confident 15% reduction.
We have 50,000 sticker submission every month and the rejection rate is around 10%.
Operation team has to call drivers whose sticker submissions are rejected and it is estimated to take 3 minutes to call
50,000 drivers * 10% rejection rate * targeted 20% reduction in rejection * 3 minutes per driver / 60 (Mins -> hours)
→ estimated 37.5 hours reduction in operation team workload
Hypothesis about the feature value
In order to make sure if we solve the problem, we set hypothesis:
If we add object recognition to the license plate in the monthly photo check, then we will reduce invalid photo submissions and eliminate approximately 50 hours of operation team workload.
Design MVP
I designed MVP for the solution.
Overall flow
As a driver with sticker, I want to have instructions on how to scan the license plate and stickerso that I understand what I need to do next while completing the tasks
To reduce technical effort and maintain a consistent user experience, we modeled this feature on a similar existing one. Animated guides walk drivers through each step, and they can reference a sample photo at any time. To avoid confusion, the instructions only mention the license plate, as our data shows drivers rarely struggle with sticker visibility.
As a driver with sticker, I want to verify my photo against the example, so that I can prevent it from being rejected and having to do it again.
I propose replacing the pre-capture document guide with a post-capture comparison step. This allows drivers to directly compare their photo against a sample image, which is more effective than asking them to recall earlier instructions.
Cognitive load theory
As a driver with sticker, I want a manual camera option, so that I'm not blocked if the AI detection has an issue.
To prevent users from getting stuck, a fallback mechanism is required in case of AI camera failure.
Assess 1st Testing
We launched the feature in 3 cities to gauge the impact.
Rejection rate with breakdown by reasons in 3 cities
The results were disappointing. While the overall rejection rate improved only marginally, rejections due to “Photo quality is low” alarmingly increased.
Define Problem to Solve
Focusing on solving the increase in “Photo quality is low”
In order to investigate the root cause of the increase holistically, I structured the issue by 4 segments: process, displays, systems, and people.
How can we validate or invalidate that the new camera design causes an issue to photo quality?
While the tech team was investigating the potential system issues, I was able to analyze the potential issues with usability.
Research goal
The goal was to:test a hypothesis that the new camera causes usability issues, which leads drivers to submit photos that don’t show sticker and license plate clearly.
The “Photo quality is low” reason is frequently selected by the operation team because it serves as a catch-all for various issues, such as blurriness, poor framing, or improper lighting.
Methodologies and key questions
We observed and asked 3 drivers in person to tested both the old and new camera flows in the app to see if there is any change in behaviors or issues.
Participant criterias:
- Drivers who failed the photo check in the testing month with the reason "photo quality is low" or "Sticker is not fully visible"
- Drivers who passed the photo check in the previous month
Key questions to answer are:
- How do drivers take photos of vehicles with the old camera and new camera?
- How different is photo-taking between the old camera and the new camera?
- What are the pain points of drivers when they take photos with the new camera?
The testing invalidated the hypothesis that the new design would cause usability issues. All participants were able to take photos without much struggle, and all resulting images met our requirements.
Other findings from the testing
01
Drivers generally ignore the captions in the old camera, appearing to rely on existing knowledge of the workflow.
02
User preference on the cameras was divided. While some drivers found the instructions in the new camera helpful, others preferred the simplicity of the old camera, which lacks a detection function.
03
No one uses zoom in and out function of the camera, they move themselves to get the right angle. Only one takes horizontally
What are the potential system-related causes of reduced image quality?
After the tech team investigations, we found out a few things related to systems that cause photo quality to be lower.
01
On older devices, the video frame can result in lower-quality photos due to hardware limitations.
02
The internal system automatically blurries
After the app captures a photo, it is sent to an internal system for temporary storage before being forwarded to a driver management portal for agents to review.
03
An aspect ratio incompatibility between the taken photos and the portal causes cropping.
To provide drivers with a larger capture area, the camera frame was set to full screen. However, this creates an issue on devices with non-standard aspect ratios (i.e., ratios other than 16:9 or 4:3). This problem is particularly prominent on Android devices, which feature a wide variety of screen dimensions.
Design 1st Iteration
In response to the system issues we identified, we implemented the following solutions.
Compress the photo compression by 80% to ensure the photo quality from the video frame and fit 5 MB file size limit.
To meet the portal's aspect ratio requirement, we changed to the 9:16 ratio from the original camera.
Hypothesis about the enhancement
If we implement the compression strategy and change the camera frame ratio, then we will see the reduction in the rejection reasons, “Photo quality is low”.
Assess 2nd Testing
This time, we launched A/B testing of the feature to a small batch of drivers in 2 smaller cities for the whole month and launched to a small batch of drivers in 1 big city for 2 weeks.
A/B testing: Rejection rate with breakdown by reasons
AI Camera improved rejection rates by 10% over the original flow, with fewer rejections due to “License plate was not fully visible”.
Comparison to previous month: Rejection rate with breakdown by reasons
Volume and rejection rate by camera type
This confirms AI camera's effectiveness in reducing rejection rates. However, over 30% of drivers utilized the fallback option when not using AI. It’s proven that AI camera can lower the rejection rate.
Define Problem to Solve
Priority is to reduce the usage of fall back option among drivers
The testing results proved AI camera's superior effectiveness, which led us to encourage its use over the fallback option.
70% of drivers opted for the fallback camera instead of continuing with the AI camera when given a choice.
After 5 seconds of failed license plate detection, a dialog appears.
Most drivers instantly select the primary button, suspecting they choose the most visually prominent option.
The 5-second timeout does not provide drivers sufficient time to to detect a license plate.
With a median completion time was 15 seconds; the 5-second timeout is insufficient. We believe the short timeout prevents successful AI camera use, forcing drivers to resort to the fallback option.
Design 2nd Iteration
Extend timeout to 10 seconds and reverse button positions to reduce fallback camera usage
We extended the insufficient 5-second timeout to 10 seconds as a balanced compromise that addresses stakeholder concerns about user frustration from longer waits.
We are also exploring whether users choose the fallback camera simply because it appears as the most visually prominent option in the dialog.
Hypothesis about the enhancement
If we increase the timeout to 10 seconds and reverse the button positions, then we expect fewer drivers will choose the fallback option.
Assess 3rd Testing
We launched a consistent month-long A/B test across all three cities, reflecting our increased confidence compared to our previous testing.
A/B testing: Rejection rate with breakdown by reasons
AI Camera outperformed the original flow by 14%, showing further improvement from previous testing while maintaining fewer rejections due to 'License plate was not fully visible.”
Comparison with previous tests: Rejection rate with breakdown by reasons
Rejection rate remained similar before the change, but the composition shifted as license plate issues decreased drastically while low photo quality problems increased.
The shift might be related to our larger driver rollout compared to the second test and warrants monitoring.
Volume and rejection rate by camera type
AI camera adoption increased while demonstrating continued effectiveness over fallback cameras. Only 19% of drivers used the fallback camera at least once, representing a 37% decrease from the previous version.
Define the final testing goal
Launch A/B testing in the rest of the operating cities to gauge the effectiveness of the feature
Different markets show different driver behaviors, so we needed to verify the feature's effectiveness across all cities. During this process, stakeholders suggested several small guide improvements, which we incorporated alongside the A/B testing.
Design 3rd Iteration
Updated the pointer guide to mention the sticker, ensuring drivers understand they need to capture both the license plate and sticker in their photos
The original guide only referenced the license plate, prompting stakeholder concerns about not specifically mentioning the sticker, which is an essential component of sticker retention requirements. However, data shows that sticker visibility has not been an issue in practice.
Added a preliminary guide that appears before drivers begin the process, providing clear instructions on proper photo-taking techniques
The local operations team expressed concerns that insufficient detailed guidance could confuse drivers. In response, we added a comprehensive guide featuring an instructional image with specific tips, similar to the original design.
Modified the dialog text that appears when the AI camera hasn't detected a license plate within 10 seconds to assist drivers in capturing a proper image with helpful tips
We leveraged the 10-second detection failure point as an opportunity to provide drivers with helpful tips, encouraging them to try again rather than immediately resorting to the fallback option
Assess 4th Testing
We implemented A/B testing across all cities, excluding the 3 original launch cities, to verify that the AI camera improves the rejection rate consistently regardless of location.
A/B testing: Rejection rate with breakdown by reasons
AI camera outperformed the original camera by 19%, demonstrating consistent effectiveness across all global markets.
Final result
We fully deployed the feature to all drivers across all cities for a one-month period.
Rejection rate with breakdown by reasons
Compared to the month before launch, we achieved precisely our target of a 15% improvement in overall rejection rates, while simultaneously reducing license plate visibility issues by a substantial 76%.
The number of drivers with stickers grew by 7,000 while the testings. We still assumed that agents take 3 mins per driver to follow up on the rejected drivers.
57,000 drivers with stickers * (8.6-10.1%) rejection rate * 3 mins per driver / 60
→ 43 agent hours reduced monthly
Next steps
We can explore the implementation of object detection technology for vehicle stickers.
Since both license plates and stickers must be clearly visible, implementing sticker-specific object detection could significantly reduce rejections categorized as 'Sticker is not fully visible,' complementing our existing success with license plate detection.
We can research more on how agents choose rejection reasons.
Now that 'Photo quality is low' has emerged as the predominant rejection reason, we need to investigate how agents are using this category. It appears to be functioning as a catch-all for various issues. We should conduct further analysis to break down the specific violations within this category, allowing us to identify and address the underlying problems more effectively.
AI camera to reduce user effort and operation time in user reward program
A driver incentive program allows drivers to display company stickers on their vehicles in exchange for enhanced rewards and priority in order allocation. Participants must submit monthly photos showing their vehicle with the visible company sticker to continue receiving program benefits.
This initiative has successfully increased brand recognition while simultaneously improving driver engagement.
Our team has optimized the verification process to minimize submission errors and reduce the time required for drivers to complete their monthly checks.
↓ 15%
Reduce in rejection rate
↓ 43hrs
Reduce in monthly agent hours
Lalamove is an on-demand delivery platform that connects 13 million users with 1.2 million driver partners in 11 markets across the world. Lalamove driver-partner app is a tool for drivers to receive and manage delivery requests from customers.
My role
- Designed the user experience and user interface
- Structured and defined the problem to solve after testing
Opportunity
Drivers aim to complete tasks efficiently, including the monthly sticker verification photo required for rewards. When rushing, they often miss critical requirements, resulting in rejected submissions. The most common issue is failing to include the license plate in the photo. When rejected, drivers must either resubmit or forfeit their program benefits.
We' explored object recognition technology to guide drivers in capturing photos that clearly show both the company sticker and license plate on their vehicles, streamlining the verification process and reducing rejected submissions.
This also decreases operational workload for local agents, who had to contact drivers with rejected submissions.
The opportunity is
How can we reduce submission rejections, helping drivers capture photos more quickly while minimizing resubmissions and decreasing the number of follow-up calls local agents need to make?
Goal
The product manager set the goal and success metrics for this feature. We want to target:
- Reduce rejected submissions
- Reduce operation time to review and call rejected drivers
- Reduce the number of rejected photos due to “License plate was not fully visible”
Steps
Understand Problem
The monthly verification requires drivers to take photos showing the company sticker from 3 different angles, with the license plate visible in each image to confirm vehicle matches registration records.
The app simply instructs what drivers need to take while taking a photo in texts.
*During initial sticker installation at the Lalamove office, the operations team guides drivers through the photo taking process, demonstrating the proper technique for monthly submissions.
Data analysis
PM and the operation team found out that main 3 reasons of the rejections are accounted for nearly 90% of the rejections.
- License plate was not fully visible
- Photo quality is low
- Sticker is not fully visible
Define Scope
Focus on “License plate was not fully visible”
We focused on our existing object recognition technology on license plate due to the high volume. While we can also detect the sticker, we chose to start with a single object to be efficient and plan to scale in the future.
Potential impact
The data shows that 40% of rejections are caused by not showing the license plate. A new feature should be designed to target this directly. While the theoretical maximum impact is a 40% reduction, we are setting our official target at a confident 15% reduction.
We have 50,000 sticker submission every month and the rejection rate is around 10%.
Operation team has to call drivers whose sticker submissions are rejected and the call is estimated to take 3 minutes per driver.
50,000 drivers * 10% rejection rate * targeted 15% reduction in rejection * 3 mins per driver / 60 (Mins → Hours)
→ estimated 37.5 hours reduction in operation team workload
Hypothesis about the feature value
In order to make sure if we solve the problem, we set hypothesis:
If we add object recognition to the license plate in the monthly photo check, then we will reduce invalid photo submissions and eliminate approximately 50 hours of operation team workload.
Design MVP
I designed MVP for the solution.
Overall flow
As a driver with sticker, I want to have instructions on how to scan the license plate and stickerso that I understand what I need to do next while completing the tasks
To reduce technical effort and maintain a consistent user experience, we modeled this feature on a similar existing one. Animated guides walk drivers through each step, and they can reference a sample photo at any time. To avoid confusion, the instructions only mention the license plate, as our data shows drivers rarely struggle with sticker visibility.
As a driver with sticker, I want to verify my photo against the example, so that I can prevent it from being rejected and having to do it again.
I propose replacing the pre-capture document guide with a post-capture comparison step. This allows drivers to directly compare their photo against a sample image, which is more effective than asking them to recall earlier instructions.
As a driver with sticker, I want a manual camera option, so that I'm not blocked if the AI detection has an issue.
To prevent users from getting stuck, a fallback mechanism is required in case of AI camera failure.
Assess 1st Testing
We launched the feature in 3 cities to gauge the impact.
Rejection rate with breakdown by reasons in 3 cities
The results were disappointing. While the overall rejection rate improved only marginally, rejections due to “Photo quality is low” alarmingly increased.
Define Problem to Solve
Focusing on solving the increase in “Photo quality is low”
In order to investigate the root cause of the increase holistically, I structured the issue by 4 segments: process, displays, systems, and people.
How can we validate or invalidate that the new camera design causes an issue to photo quality?
While the tech team was investigating the potential system issues, I was able to analyze the potential issues with usability.
Research goal
The goal was to:test a hypothesis that the new camera causes usability issues, which leads drivers to submit photos that don’t show sticker and license plate clearly.
The “Photo quality is low” reason is frequently selected by the operation team because it serves as a catch-all for various issues, such as blurriness, poor framing, or improper lighting.
Methodologies and key questions
We observed and asked 3 drivers in person to tested both the old and new camera flows in the app to see if there is any change in behaviors or issues.
Participant criteria:
- Drivers who failed the photo check in the testing month with the reason "photo quality is low" or "Sticker is not fully visible"
- Drivers who passed the photo check in the previous month
Key questions to answer are:
- How do drivers take photos of vehicles with the old camera and new camera?
- How different is photo-taking between the old camera and the new camera?
- What are the pain points of drivers when they take photos with the new camera?
The testing invalidated the hypothesis that the new design would cause usability issues. All participants were able to take photos without much struggle, and all resulting images met our requirements.
Other findings from the testing
01
Drivers generally ignore the captions in the old camera, appearing to rely on existing knowledge of the workflow.
02
User preference on the cameras was divided. While some drivers found the instructions in the new camera helpful, others preferred the simplicity of the old camera, which lacks a detection function.
03
No one uses zoom in and out function of the camera, they move themselves to get the right angle. Only one takes horizontally
What are the potential system-related causes of reduced image quality?
After the tech team investigations, we found out a few things related to systems that cause photo quality to be lower.
01
On older devices, the video frame can result in lower-quality photos due to hardware limitations.
02
The internal system automatically blurs the photo to meet the 5 MB file size limit.
After the app captures a photo, it is sent to an internal system for temporary storage before being forwarded to a driver management portal for agents to review.
03
An aspect ratio incompatibility between the taken photos and the portal causes cropping.
To provide drivers with a larger capture area, the camera frame was set to full screen. However, this creates an issue on devices with non-standard aspect ratios (i.e., ratios other than 16:9 or 4:3). This problem is particularly prominent on Android devices, which feature a wide variety of screen dimensions.
Design 1st Iteration
In response to the system issues we identified, we implemented the following solutions.
Compress the photo compression by 80% to ensure the photo quality from the video frame and fit 5 MB file size limit.
To meet the portal's aspect ratio requirement, we changed to the 9:16 ratio from the original camera.
Hypothesis about the enhancement
If we implement the compression strategy and change the camera frame ratio, then we will see the reduction in the rejection reasons, “Photo quality is low”.
Assess 2nd Testing
This time, we launched A/B testing of the feature to a small batch of drivers in 2 smaller cities for the whole month and launched to a small batch of drivers in 1 big city for 2 weeks.
A/B testing: Rejection rate with breakdown by reasons
AI Camera improved rejection rates by 10% over the original flow, with fewer rejections due to “License plate was not fully visible”.
Comparison to previous month: Rejection rate with breakdown by reasons
Volume and rejection rate by camera type
This confirms AI camera's effectiveness in reducing rejection rates. However, over 30% of drivers utilized the fallback option when not using AI. It’s proven that AI camera can lower the rejection rate.
Define Problem to Solve
Priority is to reduce the usage of fall back option among drivers
The testing results proved AI camera's superior effectiveness, which led us to encourage its use over the fallback option.
70% of drivers opted for the fallback camera instead of continuing with the AI camera when given a choice.
After 5 seconds of failed license plate detection, a dialog appears.
Most drivers instantly select the primary button, suspecting they choose the most visually prominent option.
The 5-second timeout does not provide drivers sufficient time to to detect a license plate.
With a median completion time was 15 seconds; the 5-second timeout is insufficient. We believe the short timeout prevents successful AI camera use, forcing drivers to resort to the fallback option.
Design 2nd Iteration
Extend timeout to 10 seconds and reverse button positions to reduce fallback camera usage
We extended the insufficient 5-second timeout to 10 seconds as a balanced compromise that addresses stakeholder concerns about user frustration from longer waits.
We are also exploring whether users choose the fallback camera simply because it appears as the most visually prominent option in the dialog.
Hypothesis about the enhancement
If we increase the timeout to 10 seconds and reverse the button positions, then we expect fewer drivers will choose the fallback option.
Assess 3rd Testing
We launched a consistent month-long A/B test across all three cities, reflecting our increased confidence compared to our previous testing.
A/B testing: Rejection rate with breakdown by reasons
AI Camera outperformed the original flow by 14%, showing further improvement from previous testing while maintaining fewer rejections due to 'License plate was not fully visible.”
Comparison with previous tests: Rejection rate with breakdown by reasons
Rejection rate remained similar before the change, but the composition shifted as license plate issues decreased drastically while low photo quality problems increased.
The shift might be related to our larger driver rollout compared to the second test and warrants monitoring.
Volume and rejection rate by camera type
AI camera adoption increased while demonstrating continued effectiveness over fallback cameras. Only 19% of drivers used the fallback camera at least once, representing a 37% decrease from the previous version.
Define the final testing goal
Launch A/B testing in the rest of the operating cities to gauge the effectiveness of the feature
Different markets show different driver behaviors, so we needed to verify the feature's effectiveness across all cities. During this process, stakeholders suggested several small guide improvements, which we incorporated alongside the A/B testing.
Design 3rd Iteration
Updated the pointer guide to mention the sticker, ensuring drivers understand they need to capture both the license plate and sticker in their photos
The original guide only referenced the license plate, prompting stakeholder concerns about not specifically mentioning the sticker, which is an essential component of sticker retention requirements. However, data shows that sticker visibility has not been an issue in practice.
Added a preliminary guide that appears before drivers begin the process, providing clear instructions on proper photo-taking techniques
The local operations team expressed concerns that insufficient detailed guidance could confuse drivers. In response, we added a comprehensive guide featuring an instructional image with specific tips, similar to the original design.
Modified the dialog text that appears when the AI camera hasn't detected a license plate within 10 seconds to assist drivers in capturing a proper image with helpful tips
We leveraged the 10-second detection failure point as an opportunity to provide drivers with helpful tips, encouraging them to try again rather than immediately resorting to the fallback option
Assess 4th Testing
We implemented A/B testing across all cities, excluding the 3 original launch cities, to verify that the AI camera improves the rejection rate consistently regardless of location.
A/B testing: Rejection rate with breakdown by reasons
AI camera outperformed the original camera by 19%, demonstrating consistent effectiveness across all global markets.
Final result
We fully deployed the feature to all drivers across all cities for a one-month period.
Rejection rate with breakdown by reasons
Compared to the month before launch, we achieved precisely our target of a 15% improvement in overall rejection rates, while simultaneously reducing license plate visibility issues by a substantial 76%.
The number of drivers with stickers grew by 7,000 while the testings. We still assumed that agents take 3 mins per driver to follow up on the rejected drivers.
57,000 drivers with stickers * (8.6-10.1%) rejection rate * 3 mins per driver / 60
→ 43 agent hours reduced monthly
Next steps
We can explore the implementation of object detection technology for vehicle stickers.
Since both license plates and stickers must be clearly visible, implementing sticker-specific object detection could significantly reduce rejections categorized as 'Sticker is not fully visible,' complementing our existing success with license plate detection.
We can research more on how agents choose rejection reasons.
Now that 'Photo quality is low' has emerged as the predominant rejection reason, we need to investigate how agents are using this category. It appears to be functioning as a catch-all for various issues. We should conduct further analysis to break down the specific violations within this category, allowing us to identify and address the underlying problems more effectively.
