Physical Address
304 North Cardinal St.
Dorchester Center, MA 02124
Join our daily and weekly newsletters for the latest updates and exclusive content in the industry’s leading AI coverage. Learn more
As a machine learning (ML) product manager, a simple question in my first color, inspired passionate debates between the features and leaders: How do we know if this product works? The product I managed to serve both domestic and foreign customers. The model allows our customers to identify the best problems facing the fact that they can prioritize the correct experience set to resolve customer issues. Choose from an interpocperial internet, to choose from inter-interior and foreigners The right size It was important to focus on success to seize the effects of the product.
It is like not to follow your product’s good operation, landing on air transport control without any instructions. There is no way you can make informed decisions without knowing that it is correct or wrong for your client. In addition, if you do not actively determine the dimensions, your team will determine its own reserves. The risk of being a ‘accuracy’ or ‘quality’ metric has many valuers will develop its own version, and will lead to a scenario that all does not work in the same conclusion.
For example, when I considered the main metric of our annual goal and engineering group, I immediately commented on: “But this is a work metic, we watch and remember the accuracy.”
First, determine what you want to know about your AI product
After landing for your product to determine the sizes – where do you need to start? In my experience, the complexity of work ML product It is translated to define dimensions for the model with more than one client. What do I use to measure a model of a model? It would not be enough to measure the results of the internal teams to prioritize the start of our models; The solutions made by the customer can risk the results of a very large adoption metric of solutions (if the customer does not accept a solution because they want to reach a support agent?).
Rapidly progressing to the period Great language models (LLS) – There is not only one output from one ML model, we have text answers, pictures and music. The size of the product that requires excess sizes is growing rapidly – formats, customers, tip … The list is ongoing.
Throughout all my products, when I try to get acquainted with the dimensions, my first step is to let customers know that I want to influence several main questions. To determine the right set of questions, it simplifies the correct set of measurements. Here are a few examples:
- Customer turned out? Metric for coverage
- How long did the product take to provide an exit? → Metric for delay
- Did the user love output? Metric for customer reviews, customers reception and arrest
After determining your basic questions, the next step is to identify a number of sub-questions for ‘login’ and ‘output’ signals. Output measurements are a decline in which you can measure an event that occurs. Input sizes and leading indicators can be used to identify or predict the results. See below to add the right sub-questions for laggring and leading indicators to the questions above. All questions are not necessary to have the lead / lifting indicators.
- Customer turned out? → Coverage
- How long did the product take to provide an exit? → Delay
- Did the user love output? → Customer feedback, customer setting and retention
- Does the user show that the output is true / wrong? (Exit)
- The output was good / fair? (Access)
The third and final step is to identify the method to collect sizes. Most dimensions are placed by a new device through information engineering. However, in some cases (as a 3rd question above), especially for ML-based products, you have a manual or automated assessment option that evaluates model performances. Well, it’s not fair and good, it’s not good, fair and good, not good, and it will help a rubric that will help you place surface work for a good and tested automated automated evaluation process.
Example Indications: AI Search, List Description
The above frame can be applied to any ML-based product To determine the initial sizes list for your product. Search by searching as an example.
| Ask | Metric | Metric nature |
|---|---|---|
| Customer turned out? → Coverage | Search meetings with search results shown to the customer | Speech |
| How long did the product take to provide an exit? → Delay | Time was taken to show the search results for the user | Speech |
| Did the user love output? → Customer feedback, customer setting and retention Does the user show that the output is true / wrong? (Output) output was good / fair? (Access) | % Of search sessions with search results with search results or% of search sessions with customer clicks % Of search results mentioned as ‘good / fair’ for each search duration, in quality rubric) | Speech Entrance |
About a product to create drawings for a list (whether or not a menu item in a product list on Amazon)?
| Ask | Metric | Metric nature |
|---|---|---|
| Customer turned out? → Coverage | Lists with a descriptive description | Speech |
| How long did the product take to provide an exit? → Delay | Time was taken to create drawings to the user | Speech |
| Did the user love output? → Customer feedback, customer setting and retention Does the user show that the output is true / wrong? (Output) output was good / fair? (Access) | Technical content team / Seller / Customer adjustments% of lists with generated drawings required % Of the list drawings as ‘good / fair’, in quality rubric) | Speech Entrance |
The above-mentioned approach is expanding to many ML-based products. I hope this frame helps you determine the best set of sizes for your ML model.
Sharania Rao is a group of products manager Intuit.
Source link
