Decoding Retail Data with Vivek Anand

Erin Pearson: Hey everyone! Welcome to today’s episode of Decisions Now. I’m your host, Erin Pearson, and I’m very excited because today we have Vivek Anand, who’s the Director of Advanced Analytics at Gap with us today. Vivek, it’s great to have you here. Thanks for joining.

Vivek Anand: Yeah, thank you. Thanks for having me.

EP: I’m particularly excited about some of the stuff that we have coming up in today’s episode because I think, in particular, when we’re talking about some brands like the Gap or just retail brands in general, there’s a lot that you don’t even know that goes into it when you are the consumer, and you’re the shopper as well. So, I’m particularly excited about some of the topics that we have. Just to jump right in – so, you’ve been at Gap for about a year now, and you’re doing some really cool work in data science. So, can you tell us a little bit about what that is? And you had mentioned previously that specifically you’re doing a lot for solving for demand, and that’s a very complex process involving many teams within the company as well. So, can you just talk through how do you find and solve these questions for them?

VA: Yeah, sure thing. So, in the context of fashion retail, demand forecasting becomes a unique problem, if you will, partly because of the continuous nature of the products that we introduce. If you walk in store every month, you will see a completely different store. So sometimes – not sometimes, actually, often enough – we have to predict or forecast demand for something for which we have low or no sales history. And that’s the uniqueness of retail, in general. Stuff is new, continuously coming into the stores, and you have to predict demand for it. So, for this one and pretty much every data science project that we have on our side, we follow a very systematic and collaborative approach. And first and foremost is to understand the business landscape. So typically, as data scientists, we are usually insulated from the business. So, it’s very important to understand what you’re trying to solve for. So, we try to collaborate with our stakeholders. And those stakeholders could be from merchandising, planning, store operations, and different teams were involved in it. We work with them to understand what their pain points are, what are we actually trying to solve for. Yes, demand is one problem, but it’s a bigger problem. So, we try to understand the business landscape. We try to get the expertise from different stakeholders to get a sense of the problem at hand, what are the business constraints, what are the business nuances that need to be baked into the solution. So, that is really the most important part. And then after we have done that part, we sort of go into our own wheelhouse from a data scientist standpoint, we look at the data, we try to understand, curate the data, and really try to assess, validate what we heard from our stakeholders. Many times, not just at Gap, but elsewhere as well, people come up with some sort of, I call it urban legends, like, hey, this is what happens in our business. But when you look at the data, you see something completely different, right? So, we try to understand the data. We try to validate some of the assumptions or business constraints that have been given to us. And then we really go into model building exercise. Now, model building exercise really depends on the nature of data we have. So, we try to get data from sales, customer, like similar sales, product similarity, because products are new, and also external data like seasonality and things like that. And once we have that data, we’ll go into our EDA, we’ll go into looking at the trends in the data that we have. And then finally, we go into the model selection and development phase. Now, model selection, the choice of model really depends on the data we have at hand and the problem we’re trying to solve for. So, the most important thing is demand, for example. Given the nature of the problem, like I said previously, traditional time series forecasting doesn’t work. You will have to rely on advanced machine learning techniques, reinforcement learning techniques, to come up with a solution to this problem. And we’ll just go through a host of modeling options, depending on the problem, like I said, and we’ll continue validating testing, backtesting on historical data, on historical products that we had solved for which we have a good amount of data. And then, once we have landed a candidate model, then we go into model validation testing, making sure that, hey, how’s the accuracy when testing on historical data? Because that’s the best metric for us to test. We also somehow, in some cases, we can, depending on the model, we can do some submersion or simulation, like have a chain event, like, hey, you started here, and then you forecast it depending on sales. Because in our business, there’s a lot of complexity. You forecast something, you get it wrong, then you have a lot of liable inventory sitting somewhere. So, you have to effectively simulate all the moving pieces and see how well did you do. And then once we have the model validation testing, then we collaborate with the second set of stakeholders that we have, working with MLOps, IT teams to scale the model, make it scalable, automated, and make it available in real time. In that process, the other thing that we do, like many teams, not at Gap, but elsewhere I’ve seen is they just lose track of what we call a model drift. A model’s built to do something, two years down the line, it’s not doing what it’s supposed to do. So, you have to account for that drift and make sure that model continues to behave the way it is supposed to. So that model monitoring needs to be in place as well. So, we work with, like I said, stakeholders on IT and DevOps and MLOps to professionalize the model, scale the model, make it available in real time, and monitor the drift of it. And then throughout this entire process, it’s very important to have an open line of communication with our brand stakeholders, our end customers, not people going to the stores, but people who we are trying to solve their problems for. And the important thing there is making sure that, hey, do I understand your business constraints, guardrails, rules, nuances correctly? If I understand something wrong and do not sniff test with our stakeholders as we go, then usually we end up in a not-so-nice place. So, we continuously sort of talk to our stakeholders and then make sure that what we’re building aligns with what they’re expecting or solves their pain points, if you will.

EP: Yeah, I think that’s great. What you had just said, especially at the end there, the concept applies no matter who your stakeholder is, if it’s internal or external, but it’s essentially the concept of the five whys, right? You have to ask, why are you doing this five times before you actually get to the real pain point and to make sure that you’re solving for the actual problem that they have, not just the, as you called it, I think, the urban legend that they might be saying on the surface. So, communication is key. I think it’s a skill that’s oftentimes undervalued across the board, but no matter what your job is – engineering, sales, marketing, whatever –communication is what needs to happen.

VA: More so in our space, it’s almost one of the most underappreciated skills, which is the most important in my opinion, one of the most important.

EP: So, another area that you mentioned working with is within user-generated data, such as search engines. So, how is AI involved in this process with the search engines or even computer vision? And then with the increase of generative AI over this past year, where do you see that this could be applied to take this even further?

VA: Oh, yeah. I mean, search data, it can be organic search on your own website, or it could be like a search trend on Google or whatever. It’s an invaluable source of critical information, really, for us. And it can provide information on multiple dimensions. It could be on customer, on product dimension, on even website functionality, really. So, I mean, the search data can be used to understand customer preferences and then really develop new products. So basically, a good example is, if you just analyze the search queries, you will sort of get a good sense of retail trends. For example, people are looking for, let’s just say, pink floral dress. And then it can get to you a sense of, hey, this is in design, this is what is trending. And do I have that in my assortment? Is there a product gap in my offerings? So that search data, if analyzed correctly, you can even try to understand what is the trend, what people are looking for, and what I’m offering. Is there a gap? And try to fill that gap. And essentially, it can make your process to bring new products, which are in style, in stock pretty quickly, effectively by analyzing the emerging fashion trends in, effectively, real-time, right? It can also help you stay ahead of the competition by adapting to customer preferences, incorporating popular trends. To your point about Gen AI, it can be a source of inspiration for new product development. Actually, I know somewhere people are using these things to develop new products. And you can, like I talked about, identify gaps. There are courses on websites like Coursera or Udemy, I don’t remember which one, which essentially tells you how you could use a Gen AI like Midjourney to create new designs and have your own business. For example, I forget which brand it was, but essentially, there was a search I used that was, ‘pink dress, pink floral dress.’ Someone created a design which was effectively what a vendor was selling. So, probably they are relying on Gen AI to create those products. So, it’s a new realm, but I can clearly see how it does on the product dimension. Search can also be useful for identifying customer behavior. So basically, if you are tracking, for the most part, if for any e-commerce or a company that does business online, there is a way for you to identify who the customer is, and there are multiple ways vendors do that. And then, just tracking the clickstream, you can identify what kind of product that person is in. And then you can generate tailored product recommendations, promotions for that particular customer that enhances their customer experience, right? And then the other thing, which is quite important and underappreciated, is making it easy for people to find stuff on your website, right? Just following the clickstream data and search data, like what people search and where they went on the website, it sort of helps you even optimize your website, making it easy for people to search what you’re offering on your website in a relatively low amount of time, and then reduces the abandoned cards or disrupted sessions, if you will. So, I mean, search, like I said, provides information on product, customer website functionality, like multiple dimensions. It just gives you a lot of unstructured data but, used correctly, it has a huge potential for sure.

EP: So, I mean, I think Amazon is the classic example of this, right, is like being able to see what is somebody searching, and then they personalize the different results. And I would have to imagine within Gap or other retail companies as well, whether it’s being personalized to the individual or to a region as well, I think…I mean, even when you go into brick-and-mortar stores, I get the sense that every now and again, they change up where they’re positioning different things, not only based on where people go, but also to make sure that they find new items. And so that then they can start finding them, buying them, and making more revenue streams off of these new items, just because if you walk in, if you always go in the same direction, you’re going to buy the same things. Switch it up on somebody, and they’re going to find something new.

VA: Oh yeah, I mean, it’s not just you’re thinking. It’s actually happened. So, there is a study where like…there was a theory that women buy cards more than men. And then what they started doing is they ran an experiment. I don’t remember what, this was like in some different country, probably in the UK. There’s a study on it, I can share that. Basically, what they started doing is putting a greeting card next to the beer aisles and men buying greeting cards started increasing. So just moving stuff around, you can just increase your revenue stream, like you said.

EP: Yeah, it’s smart, and you have to figure out where your buyer’s going, where are they looking.

VA: Yeah, yeah, the mind association rules for product placement and even cross-selling, like Amazon you mentioned. If you’re buying a monitor, it will say, oh, do you want the cable or an HDMI cable or something? It’s just all data science in the background.

EP: Yeah, yeah, I will say I’m definitely a sucker for data science. I will definitely buy some of those personalized items. So, one of the things that we’ve talked about a bit is measuring the impact of all of this can be traditionally quite hard. As you mentioned, there’s a lot of unstructured data that goes into it, so not everyone is able to use it very well. So, what is your process for making sure that it does get used and for measurement as a whole?

VA: Yeah, sure thing. So, I mean, typically it’s…fashion. Again, I alluded to it in the past – our outcomes are from a data science model or something. It’s just very intangible in each of our outcomes. If I do a wrong demand forecast here, in time, its impact will be felt two months down the line because I placed a wrong item in wrong quantities in the wrong store. So, those things can happen. So, it becomes more challenging. There are multiple ways we do that. So, first thing is, if I’m trying to solve a problem, let’s just say I’m saying, hey, I’m doing demand forecast. Then I need to be able to…if done correctly, you should have fewer lost sales. You should have fewer standard inventory. You should have better solve through and things like that. So, what we do is we typically track relevant KPIs before the intervention or the model that we have introduced. And after, we can just track how it is trending with the solution. That’s the most simple thing that we can really do. And again, like I said, if my solution is right, it should impact the bottom line and the associated KPIs. In the model building exercise, we typically use metrics like root mean square, or mean absolute percentage, just standard statistical metrics to understand how well the model is performing on the historical data. I can build the model on a data set and then test it on the set of data it has not seen. In traditional data science, we call it test and train split. So, you train the model on a set of data and test it on the set of data on which the model has not been trained, it is unseen to the model. And see how well it is performing. So, we do those things as well. The other thing that we measure the value is having a sort of controlled study A-B test of sorts. So, for example, hey, I want to change the outlay of the store. I want to rearrange the store in a certain way. Then you can have a similar store where you do not make that change, and in another store, you make that change. So basically, you are controlling for, since the stores are very similar statistically, all the things are similar, and the only thing that you are changing is the layout, right? In that case, that’s the live A-B test. We will see, okay, I had intervention, everything was the same. What is the difference in outcome? So, you can measure the lift from that and the statistical significance of it, which sort of lets us say like, hey, with a certain degree of confidence that we did it right or like, hey, this did not yield results kind of thing. And often enough in our line of work, where the results of our predictive models or our solutions are not just a point in time, they happen over a time horizon. We sort of have a simulation where we’ll just try to simulate or mimic how the analytic would play out in the real world, where we try to mimic different components that are fixed. So, for example, again, using demand forecast, I will look at how much inventory I have, how much of it I sent to store, how much of it sold, how much of it were losses. So, we will simulate this over a long-time horizon and see what would be the benefit or what was the observed benefit because of the simulation. So typically, standard, like I said, error metrics, simulation, A-B tests, which are pretty big, not just in retail, but even in online business and Facebook and Amazon as well. And then just tracking the relevant KPIs. Your A-B tests can give you that, hey, you have XYZ left. But if your KPIs are not tracking, probably the test was not very accurate. So, it’s a combination of all those things.

EP: Yeah, that makes sense to…I guess it depends on what you’re measuring and the timelines that you’re working on, too, for all of this. So, prior to Gap, you were working at the Zilliant data science team, and you launched the AI system around pricing that was specific to B2B. I think a lot of the people that we have listening today are in the B2B space as well. So, can you tell us a little bit about that? And what was the driving need?

VA: Yeah, absolutely. At Zilliant, I led the North American data science team as Director of Data Science there. And we sort of undertook the development and launch of AI-powered systems. So basically, like five years ago or seven years ago when AI was starting to take off, especially in the B2B space where decisions were made in isolation, there was no data culture, if you will. Or, data culture was lacking; I should not say it was not there. In traditional B2B industries, essentially the solution that we did was specifically tailored to them. And B2B, to your point, what was the need of it? B2B is very different from traditional retail. Traditional retail, usually you will have fixed prices, fixed promotions, which everyone can see. And now, you walk into the store, you see the same price. You go to the website, you see the same price that I see. In B2B, those prices are very opaque. You will be pressed to find a lumber manufacturer that will give you a price. They’ll say, hey, call us and get the price. And in that case, the pricing is dependent not just on when you are calling, it’s dependent on multiple host factors. It could be who you are, where are you calling from, who is your end customer, YOUR end customer. And it involves a lot of individual negotiation depending on the selling situation. Every unique selling situation is also at a different price. And this might seem like, hey, this is just too subjective. It’s really not. And our solution was able to bake those into the solution and build a pricing solution. Also, the other thing about B2B is, unlike traditional retail, the data generation, the amount of data generated is very low. There are hundreds of thousands of transactions compared to millions, right? So, the scale or amount of data that you generate is very low. And that essentially plays into the part where like, hey, just put a ton of data into model, have the reinforcement learning method, learn it. That is not possible. You have to get creative using data science with a limited amount of data. And that’s where novelty and ingenuity came from there. And the pressing decisions, like I said, were really dependent on contextual factors. For example, and I have seen this across industries, across multiple verticals, things are usually more expensive in New York or California compared to Florida. It can be anything, just pick something. Like pumping supplies, they’re cheaper in Florida just because there’s so much competition, anything. I mean, I’ll not go into examples. And then who the end customer is. I have worked with a customer who used to sell something that gets used in food for us – you and I, baby food, and pet food. Exactly same thing. Depending on who the end customer is, the price is different. The human – oh, you have many options, you can just buy it from anyone. So, the prices are lower. Pet food and baby food are three times, four times the multiplier for adult food. So, it’s very contextual and you have to build rules or learning mechanisms to parse this context and then apply it to the pricing algorithm. So, these were the needs that drove it. And eventually, we got to a solution that could understand these things, was not data hungry. And we were able to use advanced machine learning techniques, like I said, we’re not data hungry, and then get to a point where we were able to contextualize these, build a system that would understand these nuances, and provide the pricing.

EP: I mean, it’s certainly very interesting. I think you go into anything from what brand name of something do you buy, or if something is personalizing the price for you, or what time of day are you shopping? Anything can adjust the price on you a bit.

VA: There’s a concept called dynamic pricing, which exists in both retail and B2B. It really depends on what time of day you’re buying.

EP: Yeah, no, there’s a lot that goes into that. So, since we’re talking about pricing, pricing serves as, obviously, a crucial area of all businesses, and it’s a means to an end to achieve most of the goals, and that’s typically either increased revenue or increased profits. So, when you’ve been working with businesses, how are they navigating the decision-making process in terms of optimizing between the two, or can they be both at the same time?

VA: That is a great question, actually. A lot of people who start on this journey, I’m not sure what are they trying to solve for. They’re like, hey, I just need a price for this. But the price, like you said, it’s a means to an end, right? It’s not an end in itself. So basically, you can optimize for profit. So, just increase your price and you’ll have higher profit, or you can have a lower price, and you’ll have more sales and your revenue will be higher. So, that’s the two pricing strategies. In what selling situation or what situation do you go for revenue versus profit? Depends really on who you are, who your customer is, and what you’re trying to sell. For example, if you are a new entrant to a market, your objective is to, hey, I need to grow market share. In that case, you want to have as much sales as possible. And, in that case, you probably want to go for the optimal pricing strategy. And then if you’re someone, let’s just think of someone who has a monopoly of sorts, they built something that everyone wants – they don’t need to go for revenue optimization. They can just go for the profit. So typically, if you are in a situation where you’re an established player or you have something that everyone wants or you have effectively a monopoly, you will go for price optimization. Whereas if you are a new entrant or you’re trying to introduce a new product and you’re just trying to engage in the market, then revenue optimization is a relevant strategy. And to your point about, can I use both, typically, all the customers that I work with across multiple industries, I would even say like Fortune 50s, we had to explain that, hey, you have to go this way or this way, depending on the situation. And typically, we recommend not going full throttle in one direction versus the other. And the pricing solution I was talking about was, again, a self-learning mechanism that we’re going to understand. What is the strategy you need to take? You go profit-aggressive, and it’s not working out. It will start dialing it down to the point where you are like, okay, let’s preserve the revenue. Because at the end of the day, if you’re profit-aggressive, your pricing, increasing prices, and price elasticity comes into play if the price elasticity is applicable. In that case, your demand will start going down, and you’ll start losing the bottom line.

EP: Yeah, so that’s going into a little bit about how they’re measuring it, right? So, when they are building a strategy and you’re working with them on it, how long would you recommend that they choose revenue versus profit and go down that route? And then how are they measuring the progress to that journey before they might consider switching it up or changing their approach?

VA: That is a good question. What strategy you take really depends on, like I said, on the selling situation or who your customer is and everything. But whether you take a profit-optimizing strategy or revenue-optimizing strategy, it really goes back to science. There’s less intuition in that. Basically, what we used to do is we tried to estimate the elasticity of products you’re trying to sell. So, you have some historical information. We would segment that data, cluster that data into different selling situations, which is very similar. And we’ll look at the impact of price increases and decreases on those changes in the volume and the historical data over different periods of time to assess how sensitive this product is to price changes, which is effectively elasticity. If you’re sensitive to price changes, then there is a curve you can build, which looks like an inverted parabola. And you can identify, OK, if I am away from the optimal and I can climb towards optimal, you can try to increase prices and be more profitable. And if you’re on the other side of the profitable, like the optimal price point, then you try to walk back. So, it really depends on where you are from the optimal price point that would sort of optimize your profit or revenue or whichever thing you want. So really, I mean, it again is driven by science. And you can identify that, hey, I have started reaching a point where I’m not profitable any longer. I cannot push the profit because it’s starting to eat into my revenues. Then you dial it back. If you’re on the other side where you can increase the profit and increase revenue, then you just keep dialing it up.

EP: So, I’m running, I guess, a little bit low on time here. So, I want to make the last question, try to bring it a little bit full circle. You said in the beginning, a lot of times people come to you with a problem that’s more of an urban legend. So, they’re not necessarily using the data to back up some of the pain points. And I think that there’s a lot of trouble that people likely have trying to just get…whether it’s people to believe in the data or to adopt data-driven solutions as a whole, you’re working with a multitude of different teams, as you’ve talked about throughout this, the past 30 or so minutes. And each one of these teams is going to come in with a different mindset in terms of their history or their track record with data, their ability to use it, et cetera. So, how do you drive adoption with these teams when you do come to them with a new solution, in particular a sales team, just as we were talking about pricing over the past few minutes, but how do you come to them and say, this is going to help you in the long run, especially if they’re not quite on board?

VA: I have seen this movie multiple times across multiple industries. It’s not isolated to one or the other. So, I can give examples here. So basically, but before I give examples, a good way to think about it is the change that you’re trying to bring to the organization, is it aligned with what you’re promoting or incentivizing, right? That’s the right word. So, for example, a lot of sales commission or incentive is based on how much you sell, right? If you’re trying to incentivize like, hey, you make a hundred sales, and boom, you get the best bonus or whatever, right, commission. In that case, it’s a race to the bottom. Everyone will try to price as low as possible to just get the sales quota done and get their incentive done. And this was a problem with one of my customers in the past. And we were like, hey, this needs to change. If you want to drive adoption of the pricing strategy, two things need to happen. The first thing is you have to enforce this culture of, why am I going into the bottom? So, we built tools for our customer that could, if a salesperson is coming to you and saying, hey, I need a lower price rate, you can just show – here is your selling situation. Here are 50 different selling situations, very similar to yours. Why do you need to be the lowest one? And sometimes just very basic things, like making them write a comment, like why do you need a lower price, prevents people from doing that. The other thing that people do is they will set a target price, which is usually not the least possible price that you give, and tie your incentive index to a target price. If you’re below the target, your incentives go down. If you’re above the target, your incentives go up. So, we built that kind of system for one of our customers. One of our customers actually just made everyone write – if you want a price overwrite or to go to the bottom, why? Just give a reasoning. And usually, people draw blank and they’re like, okay, let’s just go to a different price point. So really, I mean, it’s one of the things that people struggle with, and this is not necessarily to one company or one industry. People think that, hey, machine learning, AI is coming to replace me. It’s taking my job. The idea is to convince or let people see that it’s there to make your life easier. You spend most of your time on the most complex problems. If you have vanilla problems, you can just rely on the machine to do it for you. So, it’s trying to help you. It’s not trying to replace you. So, that thing needs to be reinforced. And then anytime you want to deviate from a recommendation which is data-driven, you can have some sort of argument. You don’t have to do it for the lowest dollar, like $5 stuff. But big-ticket items, if you’re going down, you can kind of showcase like, hey, I understand you have a unique case, maybe, but here is what others have done in a similar situation. And again, it cannot be, like, leave money on the table, try to get the most out of it. And again, every selling situation can be different, but making people see that the machine is recommending something based on these things makes sense to me. It should make sense to you. So, it’s really about driving the synergy to be the man on the machine who is using it and incentivizing the most important…don’t let people overwrite a price because they can or their incentives are misaligned with the outcome you want. If you just like, these are the basics, do that and you will have adoption for sure.

EP: Yeah, well, I think that’s great. And I think it’s a great piece to end on as well. I mean, you don’t know of many businesses that year-over-year decrease their quotas or their targets. So, if you can bring AI in and the strong data science team in to help you achieve a bigger quota more efficiently, then I think it’s great. And it’s really there to help enhance everything that you’re doing and make your life a little bit easier so that you can hit those goals. Well, Vivek, it was great to have you on the podcast today. I really enjoyed the conversation. So, thank you for joining.

VA: Yeah, absolutely. I enjoyed it too, so thanks for having me.