Shameek himself has spent most of his career in driving responsible adoption of data analytics/ AI in the financial services industry. He is a member of the Bank of England’s AI Public-PrivateForum, the OECD Global Partnership on AI, and the Monetary Authority of Singapore-led Veritas initiative on Fairness, Ethics, Accountability and Transparency in AI.
Most recently, Shameek was Group Chief Data Officer at Standard Chartered Bank, where he helped the bank explore and adopt AI in multiple areas (e.g., credit, financial crime compliance, customer analytics, surveillance), and shaped the bank’s internal approach to responsible AI.
Full Transcript
Intro:
Don't start small. Start big, when it's coming to AI, right? Now, I know it sounds counterintuitive. And I don't mean spend a lot of money and build a big system. What I mean, though, is that if all you do with a new technology is solve an existing problem slightly better, then the bar of proving that your new technology is better is actually much higher. It's like, oh, I've got something which is just as good, why am I going to spend a lot more money and trust and capital, as in individual capital, personal capital arguing for this or regulatory capital trying to get the regulators? Why am I going to do that? I've got something-- And this is one reason why image recognition and NLP has moved ahead. Because they weren't alternatives, right? Whereas you'd mentioned, Moritz, credit, I mentioned financial crime. Well, you know what, there is well established statistical and rule-based algorithms for that.
Lauren:
Hi, everyone. My name is Lauren Hawker Zafer and this is Redefining AI, a tech podcast that focuses on key narratives that drive digital innovation and help people understand artificial intelligence, machine learning, insight engines, and the insights ERA.
On this episode, I've been joined by my colleague, Dr. Moritz Müller. And Mortiz heads the product team in Singapore. Now, Mortiz and I, we have differing backgrounds. So, I can really promise that this conversation does prove to be a rich one. And remember, if you like this episode, take a minute to rate it, and tell us what you enjoyed. After all, we make the show for you, and we want you to get the best from it.
So, first of all, let's welcome Moritz. Moritz, welcome. Can you take maybe five minutes to introduce yourself before we invite our special guest onto the show today?
Dr. Moritz Müller:
Hey, Lauren. First of all, thank you for inviting me to the show today with you. It's a pleasure to be back. I really appreciate what you do with your team, with Squirro for this show, because I strongly believe that education is key to whatever we do, especially nowadays in an era where everyone talks about AI. I think giving AI a bit of a meaning, a bit of a very concrete meaning, and not just speaking about it as a concept is very, very helpful.
So, a bit about myself. I'm here in Singapore. I have opened the office for Squirro in Singapore two years ago and it has been quite interesting ride during COVID times. But now I'm focusing a bit more, of course, again, on the whole AI topic. It is very interesting to see how the field has evolved, how AI has evolved in the last two years. I think COVID was actually quite helpful to a certain respect, especially when you look at it on the unstructured data side. Suddenly there was a need for companies to evolve very quickly to deal with a lot of data. Yeah, it's quite interesting.
And today we have a very interesting guest on the show. I'm very much looking forward to speaking to him, speak with you, and ask him some interesting questions.
Lauren:
Excellent. Yeah, we certainly do have a special, special guest. So, thanks Moritz for that sort of introduction. And now this episode, it's called What is Trust in AI? Trustworthy AI: Lessons of a Chief Data Officer. And it is a title that I feel obviously indicates the direction that we intend to go in today with our guest, Shameek Kundu. Shameek is the Head of Financial Services and Chief Strategy Officer at TruEra. So, welcome Shamik, it's really lovely to have you here with us.
Shameek Kundu:
Thank you so much, Lauren. It's a real pleasure to be on this podcast, particularly because I have long engagement with Squirro, ranging back several years. And it's wonderful to collaborate on this very important topic from an educational perspective.
Lauren:
Excellent. And now, Shameek, you're in Asia, you're in Singapore. And you've been there for a few years now. Can you maybe tell us what's kept you there? Let's hear a little bit about you and your journey so far?
Shameek Kundu:
Sure. So, I moved back to Asia, I suppose having grown up in India, I moved to Singapore about 12-years ago when I joined Standard Chartered Bank, where I was Group Chief Data Officer between 2014 and 2020. So, that's the professional angle.
But I suppose, you asked me, you know, what's keeping me here? A few things, I think. From a professional perspective, certainly, it's an exciting place to be. Both in terms of how forward looking some of the regulatory framework around the use of data is and how much the potential of data, both structured and unstructured. It is all around us whether it is financial inclusion in some of the ASEAN and southeast Asian countries using new types of data or whether it's pioneering in a lot of the use of facial recognition, etc. in a safe manner. I think it's always been at the forefront of technology. And more recently, it's also been at the forefront of technologically related, regulatory safeguards, frameworks as well. So yeah, if you're into data, into AI, into digital more broadly, it's a fantastic place to be.
Lauren:
It's not a place I've been yet. But Moritz obviously understands because he's there himself and it's where we have our office, Squirro, in Singapore as well.
So, let's start diving into sort of topics that you've mentioned. People who are interested in data, they're interested in AI. So, we've seen as well, Shameek that artificial intelligence is all around us, not only in Singapore but globally. And it's really rapidly transforming our livelihoods in that environment in a lot of fundamental ways.
Now, building trust in AI has therefore become quite a key concern for policymakers, for the industry, and science. Now, I mean, I think we want to find out in your own opinion, when and where did this necessity for the narrative around trust in AI come from? And my main sort of question that pertains to that would be, how far is it from people just really engaging in abstract discussions about AI theories?
Shameek Kundu:
Sure, this is a really interesting question. And probably not one I would be confident in saying I have the full answer too. But I'll have a go.
I think there are actually two different conversations here. There is a conversation about the use of AI, in what is very much the public domain. And this is, among other things, social media, search engines, e-commerce, related pricing, advertising. All of these things are, you know, entertainment, all of these things are impacting a very large number of people. And use a significant share, though not all of them also involve the use of natural language processing, image recognition, in some form or the other, and often at the bleeding edge of some of the newer machine learning techniques. That's where the experimentation often happens and is realized.
In that realm, the conversation on trust in AI came from fear. It came from the fear of, we've seen some of the ill effects, let alone AI. We've seen some of the ill effects of not knowing who's accessing what data and doing what with it. We've seen the effects of that. And before horse bolts this time, let's make sure that we build the framework so that similar misuse of these newer algorithmic techniques does not happen.
Now, one can argue whether the horse has bolted, but that's certainly that angle of fear. And I say that fear because there is some genuine reason there. But unfortunately, that means that some genuine fear is also combined with a level of kind of exaggerated fear for things that are not even on the horizon, right? So, that's one block.
There's a separate block of largely traditional enterprises, banks, insurers, pharma companies, industrial firms, etc., who have a different trust challenge in AI altogether. Yes, they do have some of these concerns about fairness, about, you know, not misusing customer data, and so on. But the bigger concern, as an executive at a steel plant once told me, he said, Shameek, you worry about fairness and what social can do. Imagine if I use machine learning to set the temperature for blast furnace and that goes wrong? Imagine the cost of that, right? So, this is a different fear. It's not necessarily a fear, it's I'm not going to use AI in its fullest sense until I have the trust that this works at least as well as my alternatives. So, this one, whether you call it a fear or not, this requirement is much more one off. I need to be able to prove and test that this thing works as well as my alternatives. I do have alternatives, which may not be as complex or might use other techniques. So, there are these two conversations.
Now to your question on timing, I think the first conversation started four or five years ago. I don't know where to draw the line, but somewhere between Cambridge Analytical and some of the issues around Facebook, and all of those fears got combined in one and the additional surveillance around the pandemic, probably. Data, AI, algorithms, mutant algorithms in the words of the British Prime Minister, they all became one big blob. And that's been around for five years.
This other conversation on the enterprise side, it's a newer one. It's a much more subtle conversation. This is trust, not about, you're not going to do unfair things to protected segments along. It is much more fundamentally. I'm not going to use this stuff unless I'm confident that it is better than what I have. This conversation is only now maturing, I would say in the last two years. And people are still feeling their way around it. Around how best to draw a parallel between this and traditional software testing, for example. Because at the end of the day, machine learning provides software. So, we do software testing, how's this different? How does this fit with software certification, similar aspects? And so, that conversation is probably the last two years.
Lauren:
Maybe a question following up in that, and especially one that pertains to more the narrative or conversation that is happening in traditional enterprises. Who's involved in that conversation? And who's not involved? And maybe, who should be involved?
Shameek Kundu:
Yeah, it differs by organization. I know in one bank, the trustworthy AI group started with a monthly meeting of 10 people. And at least count, it had 150 people attending from across the bank. That is not great. That basically means a lot of people are worried and they are just trying to make sure they are there because they might miss something. But generally speaking, who's at the conversation, I think in some of the more mature organizations, everybody who needs to be. So, in the industry that I know well, banking, who are at the table, obviously, the analytics teams, the people who are building the models. The model sponsors, those who are commissioning, like, if you are a retail bank head and you are starting a new algorithmic lending program, then you want to make sure that your model is working well. The specific control functions that are designed to address data and model risk, these are two different organizations in the bank often. They're involved. Sometimes compliance organizations focused on what in banking is called customer conduct, which is how do you treat customers. Do we treat them fairly? To some extent, security people, and so on. So, all of these parties get involved, sometimes ESG, or corporate communications people just to focus on that aspect, what do we communicate to the rest of the world? And I think most of these are at the table.
The one group that I think was not at the table until recently, and they have come onboard now, is actually the technology platform owners. Because for a long time, they have focused on the efficiency of the process to build models, deploy models, implement them, monitor them. This particular aspect of testing the quality of it has been something, well, before you bring the thing into production, you would have tested it. But that's not how it works in a large scale, industrialized world. But I'm very pleased to say actually, increasingly, our conversations at TruEra now, are more and more with these platform owners. These are technology architects, technology delivery people, sometimes business people who are heads of AI centers of excellence, and have the task of giving a common platform to everybody else. And now, those individuals are realizing, you know what, if we leave this to a lot of human beings to figure out, you will have 150 plus in meetings, we need to build this in an automated fashion so that the considerations get addressed the same way software testing gets addressed.
Lauren:
Of course. One thing that might be key to the conversation, and I don't necessarily want to make a polarity between fear and trust, but we're talking heavily about trust and trustworthiness and trustworthy and AI or trustworthy AI. And how would you personally define trust in AI? Just so that we can baseline the conversation and ensure that our listeners attach themselves to your own definition?
Shameek Kundu:
Yeah. So, I would say, at its basic, does it do what it says on the tin, number one? But then, number two, derivative question, does it actually say on the tin what it does?
Lauren:
I was going to ask--
Shameek Kundu:
Right. So, these two probably are the basic piece. And because it is machine learning, it's also, does it continue to do over time what it said it would do? And also, another variant, does it do so for every group to which it is applied? So, it might have been saying, I will do x, y, z for this group of people. But now you might have applied it to somewhere else and so, can that be applied? If you cover these four, rather foxy questions, I think you will cover every aspect of ethics, bias, and everything else. Because more often than not, the reasons people find mutant algorithms is not because somebody-- I mean, I do not know a single person who went in and said, I'm going to design a really sexist or racist algorithm, right? It's ridiculous. Even a sexist or racist person will not say, I'm going to go to the trouble of designing one, right? What I have seen is people having no clue what the original model was meant to do, or where it was meant to be used, with what limitations. And now going away and applying it somewhere else without actually adjusting.
I mean, I'll give you a simple example, facial recognition, East Asian country. Bought the model from another East Asian country, which was racially very uniform. Almost all the people were East Asians. Introduced in a city with much more of Caucasian and South Asian mix, suddenly not working as well, right? They didn't set out to be racist. They just didn't realize that thing cannot be transferred straight away. So, let's retrain the model a little bit with this. And that was it, no grand designs of, let's do a really bad job of making South Asians and Caucasians unhappy. Just, we didn't think about the transform.
So, coming back, does it do what it says on the tin? And indeed, does it say on the tin what it does? And does it continue to do so over time? And over different groups to which it is applied?
Dr. Moritz Müller:
Understood. So, question, I always try to understand in terms of trustworthiness what are the drivers? I think the biggest worry on my side when I see this is not necessarily having a bias, you will always end up having biases, right? Even if you try to avoid them, there might be one and then you start correcting them. I think it's very much a question of interpretation. The moment you have something out that actually works perfectly well for 99% of the cases, and has 1% bias, and that goes into the news, you have a huge reputational risk. And I think that is, for me, from my perspective, if you look at compliance and the corporate point of view, this is probably the biggest worry that there is, that you run into a situation where you have a public scandal. And it might just affect one person or a million, but this something to be avoided because it will also avoid going forward adoption of AI. And no matter what we do, we will adopt AI solutions more and more. There is no alternative to this, I think we agree on it.
Now, my question to you is a bit, if you look at that trustworthiness, you have the corporate perspective. And for me personally, I'm happy to hear your perspective you have much more experience there. And it's not only reputational risk, potentially other affects, and components. And then, on the other side, you have the regulator tried to set a framework. Where do you see some of the key differences and motivations when you look at the regulator defining some of those trustworthiness frameworks, let's call it that way? Compared to the motivation on the corporate side, only publicity risk? What else is driving these people to define those findings?
Shameek Kundu:
Yeah. I think you've asked-- You've made a very important distinction. And it kind of goes back to that fear versus trust thing that we started with, right? I don't think though that the split is really between regulators and corporates. I think the split is between people who fully understand what this beast is and what its full set of results are. Versus those who have mostly worried about it superficially and therefore think only about that one article about Apple or something like that. Don't get me wrong, there's a lot of unfair bias out there. But the logical conclusion from a large defensive enterprise will be, well, if it is risky, and we can't do anything about it, let's just not use it.
And you said, well, there is no alternative. Actually, there is an alternative. They will use humans. They will use, in many cases, traditional models work well as well, right? I mean, some places, they can't. In NLP image, they can't. But in many other areas they will. So, I think there is a little bit of that split that you mentioned. But it is not between regulators and enterprises. It is between those who understood AI quality as the holistic problem, just like software quality was a problem years ago. Versus those who are still looking superficially at this ethics and bias issue only. The individuals and the regulators and the corporations that have understood this problem as a holistic problem around performance, stability, fairness, privacy, security, robustness, or fallback, all of these aspects, that's one group. And the other group is primarily those who are like, oh, this is all about reputation. Because then they are not understanding the real problem.
Now, with that in mind, I don't actually think there's a big difference between whether regulators minds are, and I do work, as you probably know, quite a few regulators across Singapore and U.K., and now a little bit in the U.S. I actually think the leading regulators and the leading adopters of AI are exactly in the same place, right? There are differences in terms of kind of supervision, in terms of how deeply the regulator might want to give out the question paper, so to say. But fundamentally, if you look at the UAE law, for example, the draft, it is not talking only about ethics. It is actually talking about all those aspects of quality. And so, I think the difference is not necessarily their Moritz, I think the difference exists but it's between people who understood the problem fully versus people who just scratched the surface.
Dr. Moritz Müller:
Thanks a lot. I completely see where you're coming from, Shameek. So, that's why we invited you today and we very much appreciate your view on this. And of course, I think you're right. It was more of a tricky question to look at the motivation.
Now, you are at TruEra, you are committed of course to building trustworthy AI. And you mentioned already a few factors that are part of a trustworthy AI solution. But are there one or two components that you think can potentially be forgotten and we should remind people to look in and guarantee that they have certain procedures in place to honor those when they build an AI solutions?
Shameek Kundu:
Yeah. And it's a really good question. So, let start with the ones that everybody agrees upon. I think everybody agrees upon some degree of, obviously, performance, as in, you know, predictive ability, how accurate it is in testing, but also on an ongoing basis. I think everybody now agrees on picking the fairness aspects. Although there's less agreement on which ratios to use, which is a whole area of science by itself. A lot of people are agreed about the need to respect privacy while using AI. So, these are all common agreements.
There are two areas that I think are like second, third order pieces. One is the use of the AI human interface. So, let me give an example. Often people will say, oh, don't worry, I have a safeguard for that particular model. There is a human being who's reviewing the results of the model and deciding whether to go ahead with it or not. Now, I've run many of these projects and systems. 200,000 instances, 20 instances when the human supervisor disagreed with the machine. Now, what happens when those kinds of numbers happen? After a while, you do what that Tesla driver who had the accident on autopilot happened. So, that area of, oh, I'm saying I'm giving the human power. But actually, am I? The human is slowly, slowly getting lulled into thinking, this thing knows what it's doing, why am I even bothering? So, that's an area that I don't think people have cracked. There is a lot of research going into this. But that is one area.
The other area, Moritz, is interplay of models. And I think your company's work will be quite relevant. Suppose you use Squirro to generate insights from unstructured data. And that might be good enough for a human relationship manager or a human expert to review and say, okay, you've just saved me work for reading. But now I'm going to do my own things. That's one angle. But there's a completely different angle. If the data that you've generated, structured data, generated from your unstructured text now gets fed into something else, which then gets fed into something else, and before you know it, you have a chain of 3, 4, 5 models. And those interdependencies, I think, some companies have started talking about it in their internal policies. But I don't think that's been fully understood.
And by the way, neither of these problems, Moritz, neither this one nor the previous one on human interaction is actually unique to AI. Because you can have the same issues with statistical models. You can have it with very complex rule-based models. Where it's so complex, I don't understand, machine is saying approve this, I'm going to approve. Or, well, that machine had a rule book this thick. I added another rule book this thick. By the time I go to the third machine, there is no way of knowing what is happening. So, I think those second order complexities are less well understood.
One that is somewhere in between is the interplay between security and trust. I mean, trust, implicitly includes security. But we increasingly have situations where people are worried about things like data poisoning. So, security breaches that impact the outcome of the AI, particularly with self-learning models. I mean, there are people looking at it, but it is still not that well understood, I would say, that particular area.
Lauren:
One thing that we've not touched upon there is maybe the actual data that's used. Do you think there's a difference in explainable AI when it comes to maybe the data set used of unstructured or structured? Is there anything that leaks to or that is critical for maybe more fairness or bias reduction in that sense?
Shameek Kundu:
I mean, I think that-- First of all, it's very apt to think about data because ultimately-- I mean, in theory, you could get a model wrong, both due to design choices you make, like technical design choices, hyper parameter choices, etc. As well as data problems. In practice, we find the vast majority of issues happen because of data problems. So, first of all, it is right to focus on data. Thanks for bringing us back to the Lauren.
To your question, what are the differences between unstructured and structured data? I think the advantage with structured data, particularly if it is data that was already inside your enterprise boundaries, is that you know roughly what that data is. So, you know, how to assess whether that data is good or bad and therefore you are able to put a little more control around what is feeding the engine. The problem with unstructured data is even if you've had it in your organization for a long time, you have no conceptual framework on how to decide whether that data is good enough or not. And I think that is the real problem. The question is, the exam question is the same in both cases, make sure the data is presentative and high quality, which might mean everything from complete, no dummies, and so on. But the way in which you implement it, the tool sets for managing data quality and data representativeness are far better established, in my view, in traditional structured data than with unstructured data. And that is where I think the challenge, but also the value of vendors like Squirro come in because it's an area that hasn't received the same amount of focus. So, you're solving as much a data problem as you're solving an AI problem. Whereas with a lot of the structured data, the data problems somebody else is solving, the software is just solving the AI problem.
Dr. Moritz Müller:
I think it's very good point that you make Shameek. And maybe just to reflect on how we have experienced this, you mentioned it before, and I think that it's spot on. There are different layers of AI. When you talk about layers, if you start to apply AI on different levels, then later on you combine those prospective information or outputs of the model into something bigger. That is potentially where it is going. We have seen this when we do our analytics, independently what models we use. If we use a rule-base model, or NLP models, we can easily detect interesting components to show, for instance, a relationship manager saying there is a sentence that talks about potential capital raise or an M&A event. We can do this. And if you show this to a relationship manager or someone that reads and still makes an educated decision, it is something that is useful. Can I take that forward? Can I take that up? I think in this case, the risk is limited, the risk of having trustworthy components limited because user always understands potentially why this is now highlighted. If it is useful or not, that is another question.
But if you combine this later on, let's say you take, I mean, call those signals or insights that we extract from this. If you combine this later on that you have two of those and two of those insight signals, you look at the cash funds, from a client and the cash balance available, and you start doing predictions or recommendations, this is the reason why you come into a field where you need to be careful. Because at the end of the day, you need to explain it. And for me, the explainable part, you need to actually explain what are the underlying data points and you need to explain also the model in that sense. You just need to say, look, what I had were two text indications and one structure data point indications. We have set up, very often doesn't even need to be I-registered, rule-based model on top of something that is machine learning, that pinpoints it then into the direction to saying, bring this up as a recommendation or do not bring this up. And as long as you can explain it, I think that's helpful. But very often, for simplicity reasons or because it's also overwhelming to really understand the whole value chain there, belief is out. And for me, this is where the risk comes into play.
So, maybe to add to the question to you, Shameek, have you seen an AI project that has failed because the value chain of AI was not transparent enough?
Shameek: I think I've seen tons of AI projects fails. But I'm not sure they will put the reason as that, you see. They will put the reason as, the performance went down. It was good at the start but then it went down. Or you know, it was good, but the data quality was bad over time, and that's why I didn't use it, right? Or occasionally, they might say, it was too complex, right?
But really, if you go into each of those, there is an element of this trustworthiness. They just wouldn't necessarily call it trustworthiness. So, why did the model go bad over time? Well, if you had set up good monitoring of the drift, and the root causes analysis of the drift, then you'd know why it was going and you'd have that warning, and you would self-correct or put up a red flag and say it didn't work, right? There was one instance in a company I know where the model worked very well. It was a self-learning model. So, in three months, all the easy hanging fruit was gone. And it was like, now, we don't need it. The rule-based model is very good. It only became good because of the learning from the first piece, right? So, that might be fair.
So, to your question, have I seen AI projects fail? Not just me, I think there is a lot of literature and surveys out there about the percentage of AI projects. Some people say 75%, some say higher, that not necessarily fail but fail to scale up. I think, downright failures have happened. There is a, I think, well known article or Meta article on how bad machine learning models were in predicting things around COVID. I think there was like a Meta study done on that. There is, of course before that, with one of the major tech firms, this whole piece around how bad it was for cancer detection. So, there are some big instances of failure. But I would say there is a thousand instances of failure to scale up. And that failure to scale up comes because what might work in the context of a small project with three people who are all looking at the topic, does not work when you're trying to scale it up to 300 people. And those people don't have the same knowledge, same context. Now, you want to industrialize it. That is where I think a lot of the failure happens.
Dr. Moritz Müller:
Very good point. When you evaluate an AI project, or risk AI project, I think that's what you do at TruEra as well, you advise on potential risks. How do you evaluate, in that sense, the complexity? Is complexity a key component of the risk evaluation that you do?
Shameek Kundu:
It is. Just to clarify, we don't actually provide advice. But yes, our software does the testing which can then be used to draw insights. It is and it isn't. What I mean by that, in my career now, it's a long time ago, I have been in a situation where we got fined several hundred million dollars for a very simple fuzzy logical model. Nothing to do with AI. Not complex at all. And then, I know of deep neural networks that I have worked with remarkable accuracy. So, I'm slightly wary when you say complexity equal to bad, right? But I think complexity left as complex, without an attempt to test it, without an attempt to, how do I say, abstract out what it's doing, right? You don't need to understand exactly how the image recognition model works. As long as you have such robust testing, I would test it every day against different populations. I'm not worried.
So, I think complexity in itself is not bad. But complexity left increasingly as a black box, which has not been answered, and is not understood, and is not tested, that is bad. Unchallenged complexity is bad. But complexity itself is not bad. And I think there is evidence of that, right? I mean, there's evidence of how most recently, I saw an example of people-- In February, this year in the U.S., I think one of the regulators said, oh, we're very worried about algorithms miss-valuing, if that's a word, or under valuing certain properties for mortgage assessment because of race and other consideration. And then, two days ago, I read this article from New York Times, which I don't know the factual accuracy of it. But from what it looks like, there was a human evaluator who did exactly that with a black family evaluated at 450k. The guy got a white colleague and their family to come and stay in the house and remove some evidence of the race. And then suddenly, apparently became 750k. Now, the reason I give that example is, a potentially complex machine learning model, with a lot more data points, might have been able to do a better job than that very simple option, let's just give it to an evaluator. So, complexity does not equal bad as long as you have the ability to explain and put safeguards on it.
Dr. Moritz Müller:
I think you're spot on there. That was also my point, right? I was hinting at complexity being a risk for a project. I mean, from my experience, and that goes more with the perspective of cognitive understanding of the solution, I have seen for instance, banks working in computing with high credit risk scores and for these credit risk scores it's really key that they are trustworthy and explainable. Because what I've seen in a project, a corporate bank has invested million in this and the end of the day their product life to the corporate bankers. And they ended up not using it because they did not understand it. They didn't agree with it. They had a feeling for the client. They did not agree with the particular scoring because their feeling was not in line with what the machine predicted. And that's what I mean by this complexity. That there is a certain risk when you start doing this. If the adoption is not there, it is much more the risk of the stakeholders if the adoption is not there. It is a big investment risk that you have.
Shameek Kundu:
Yes. There is absolutely big investment risk. And so, maybe I will stop with that. And then, move on to the next question. Somebody asked me recently, what do you think is the biggest risk in this AI space? And I said, the biggest risk in the AI space is people not using AI at all at scale because of the fear of all the rest of AI. Because I think there is that angle. It is for that reason that we need to crack this problem. Because if you don't crack this problem, you may get into another AI winter, over time, certainly in traditional enterprises.
Lauren:
Everything has just been so fascinating. I'm learning so much myself. And I actually wanted to ask you as well, just the same question that Moritz said, brought up there around the cognitive understanding and obviously challenging the human and building up trust in the human. But I think what I want to do before we sort of usher towards the close of our conversation is, Shameek obviously you have had a lot of experience and we've seen that displayed in our conversation today. I mean, you were the Chief Data Officer at Standard Chartered Bank. You mentioned that at the start for many years. And actually, probably the longest serving, I think, CDO in a global systematically important bank. And as you personally claim, from what I've read, long enough to see both the hits and misses of your own and your team's efforts, and I think your role was sort of full range across data strategy, architecture, technology, governance, curation, data dictionaries, meta data standards, golden sources, and analytics enablement. 2014 to 2020. So, it was obviously an interesting time to be a CDO.
Maybe as parting thought, what are your two top lessons learned from that period that we can really share first hand with our listeners?
Shameek Kundu:
Sure. I mean, first of all, before I share that. I said, I think, yes, I did claim enough to see the hits and the misses. I would say that there were more misses than hits, for sure. Yeah, you asked for two and I could give you 10 things you shouldn't do. But let me focus on two.
And one of them is counter to what, actually both of them are countered to what I started off with, but I've come to the reverse conclusion. The first is, don't start small, start big when it's coming to AI, right? I know it sounds counterintuitive, and I don't mean spend a lot of money and build a big system. What I mean, though, is that if all you do with a new technology is solve an existing problem slightly better, then the bar of proving that your new technology is better is actually much higher. It's like, oh, I've got something which is just as good, why am I going to spend a lot more money, and trust, and capital, individual capital, personal capital arguing for this or regulatory capital trying to get the regulators. Why am I going to do that? I have something-- And this is one reason why image recognition and NLP has moved ahead, there were no alternatives, right? Whereas, Moritz, you mentioned credit. I mentioned financial crime. Well, you know what, there is well established statistical and rule-based algorithms for that. So, that is my first takeaway. If you are really trying to demonstrate the power of AI and kind of really get an organization to adopt that, then the traditional logic of, let's start small and get a few wins and then we'll get there. It sounds attractive. But then, five years later all you have is a few small wins, right? Which is not helpful. That's number one.
I think the number two is very appropriate for this conversation. Is the power of education, it's too light of word. But it's the power of taking the broader organization along with you. When I started as CDO, Lauren and Moritz, I had a sum total of one staff member, one. When I finished in 2020, I had 2,500. Okay. Now I quote that, because this is an area that has grown enormously, both in terms of dollars and people and interest and interest from boards, regulators, etc. And all these people are at different levels of capability, I want to say. But also different levels of understanding. You might think or I might think as a data professional I know everything there is to know about data, but maybe the person I should be listening to is not a data professional. Maybe I should be listening to somebody like you said Moritz, a relationship manager. Data is useless until I can use it. So, I would say my second big less and my mistake, if you will, the flip side of that, was I focused too much on saying, let's get to a technocratic solution and get the right solution. I mean, so if I bring it back to the AI trusting. I am 100% confident that anybody who understands data and machine learning will quickly be able to understand that this model is reliable. But that counts for nothing. If the other 99 people who are meant to use the model, pay for the model, swear by the model in front of the regulator's, audit the model, get impacted by the model as a customer, if those people are not convinced, what's the point of me and my technocratic friends being convinced, right? And so, that would be my second big lesson. Which is, do not underestimate the power of taking the rest of the organization, your staff, your colleagues, your partners, all of them. And indeed, ultimately your customers. Do not underestimate the power of taking them along if you want to, you know, unleash the full power of AI.
Lauren:
Definitely, we fully agree with you. And you see that there's a lot of conversations obviously being ushered around, you know, the people in digital transformation and taking the brother organization with you. But yet, you still see high statistics around those that are still failing to take the organization with them. So, where is the blocker? Or why-- What's the end capability around really being able to take-- Does it pertain to the size or the maturity of an organization?
Shameek Kundu:
Yeah. It's a tough one.
Lauren:
Very much so, yeah.
Shameek Kundu:
Yeah. I would say the biggest single reason is, it is a hard problem. And it's hard not because of some mathematical difficulty. It's hard, because-- What makes it hard? It makes it hard-- A few things make it hard. One, as I just mentioned, you are trying to take a large organization along with you on a relatively technical topic, when everybody is at very different levels of understanding. The same is not true, let's say for remote working. Everyone has a say on remote working. Well, everyone may or may not have a say, as an example, I'm just thinking, right? Ways of working is something that everyone can contribute to meaningful without having to do a lot of technical reading up, etc. So, one thing is the differing levels of knowledge, awareness, mindset across the organization.
But the other thing, Lauren, it's also, I think, the lack of maturity of, of data management as a discipline. It's only recently that data management frameworks have come about. It's only recently that, you know, vendors like yourselves have kind of taken a more systematic approach, for example, to unstructured data. And I think it's fair to say, both yourself and us, we're still learning. We are still putting the framework together. You compare that with something like software engineering, which is now much more mature. The thinking is also not matured. So, I would say, these two reasons come to mind. But I doubt that they are the only ones. I think data is all around you. It's pervasive. And so, to say, we will get everybody aligned on something that, on which everybody might have an opinion, is always going to be difficult. But yeah, more systematic approach towards training as well as more automated mechanisms. Recognizing that doing all these things in isolation in a small cottage industry is not going to work. You need to do it in a way that is scaling up. I think those things will help. But will it solve the problem? I don't know.
When I started as CDO, Lauren, I said we won't have a CDO role in the future because you no longer have a Chief Electricity Officer, do you, after all these years of having electricity? I am only seeing the number of Chief Data Officers increasing every day. So, I want to say, I'm pessimistic about that. Although another part of me says, it's great. May the tribe of CDOs grow, is what I would say. But at the same time, it does mean the problem is not yet cracked by any means.
Lauren:
Yeah, it's an ongoing one. And it's something that certainly pipe in the sense of education can only enhance the understanding from every angle. It's been wonderful. And I'm sure, Moritz, you will join me in sort of thanking Shameek today for his contribution.
Dr. Moritz Müller:
Absolutely. It was fantastic.
Shameek Kundu:
It has been a delight. Same to you. Sorry, please, please.
Dr. Moritz Müller:
Thank you. I said, we could be ongoing for a while and share some stories and learnings. And I think you brought it very much to the point. There is so much data out there that the only way to tackle this going forward is technology, and in that sense, AI, whatever that means. And it's a hard nut to crack, but it's definitely worth trying. So, we are very happy to be part of that whole machine learning, that being part of one of the guns that is aimed at the big amount of data out there. It might be a bumpy ride, but at the end of the day, I'm pretty sure there is a good view of all that we do. And there is huge potential standardizing it. And I think you and I and us in general are part of this journey. And it will be interesting to see how this evolves in the next five to 10 years.
Shameek Kundu:
Absolutely. So, I can only wish you that Squirro's gun keeps firing away. And I can only thank both of you for this wonderful opportunity. Thank you so much.
Lauren:
Thank you. So, I want to-- thank you. I want to thank everyone else for listening today. It's really been a wonderfully insightful conversation. If you want to learn more about AI and ML, come and take one of our free courses at Learn.Squirro.com. Thank you.
