This episode is brought to you by Portrait.
It's the AI research system that I used to prepare for today's episode and for all Business Breakdowns episodes.
Portrait was built by former buy side investors and they understand great investing isn't just about having more information from low quality sources.
It's about having the right information organized the right way.
And if you listen to the show, you appreciate diligence consists of many things.
Diving into the history of a business, framing the nuanced competitive dynamics, tracking key signposts around your thesis.
And historically, that would take up material time that you do not have.
But portrait is basically like adding an army of analysts to your team.
It's powered by an AI system specifically designed for investment research workflows.
So you get nuanced idea generation.
Portrait Assess is the same types of qualitative attributes that we discuss on this show.
And that can help identify businesses which fit your frameworks.
Portrait also customizes research report generation.
And I use Portrait to generate a primer and lay out full bear cases ahead of today's episode to help frame the conversation.
And third, there's intelligent thesis monitoring.
And that's where Portrait assesses thousands of data points across value chains each day, extracting the insights driving the business.
Again, all this work would typically take hours and hours and hours.
It's at your fingertips now.
Visit portraitresearch.com to start your free trial today.
This is Business Breakdowns.
Business Breakdowns is a series of conversations with investors and operators diving deep into a single business.
For each business.
We explore its history, its business model, its competitive advantages and what makes it tick.
We believe every business has lessons and secrets that investors and operators can learn from, and we are here to bring them to you.
To find more episodes of Breakdowns, check out joincolossus.com.
All opinions expressed by hosts and podcast guests are solely their own opinions.
Hosts podcast guests, their employers or affiliates may maintain positions in the securities discussed in this podcast.
This podcast is for informational purposes only and should not be relied upon as a basis for investment decisions.
This is Matt Russell, and today we have a special episode breaking down how investors are using AI.
This is a question I get from many of you.
And while there is no shortage of content on the implications of AI, I know there's an appetite to learn more about tangible use cases, how to make sure you're getting the most out of these tools, how to think about advancements in the technology and ensuring that you're keeping pace with the innovation curve.
So my guest today is David Plon, founder of Portrait Analytics.
Now, David and Portrait have been a partner of Business Breakdown since last year.
But I specifically asked David to do this episode because one he is really front and center to how investors are using and applying AI.
But two, and maybe more importantly, him and his team come with a background in investing.
So, while the conversation doesn't really focus on portrait, you'll hear references to what him and his team are building and how they shaped it for investors.
You'll very much understand, when you hear David talk, that he is someone who understands the pain points of an investor.
I think everyone will find something in this episode that will benefit them in their day to day.
And I would love to hear the feedback.
Now on to the episode.
Thank you so much, David.
Use cases some of the skill sets that you can use to leverage the most out of the technology.
But I actually just wanted to start out with your background, because I think this is really unique and can speak to why I find you such a valuable resource when it comes to thinking about this through the investor mindset.
So can you bring us back in time prior to Portrait, what you were doing and how that led up to starting this business?
Yeah, absolutely.
Really a pleasure to be on here.
So I spent my career as an investor in a few different seats before starting Portrait.
I started at Barclays on the trading floor in the special situations group, was then at a long short hedge fund called Slate Path Capital, where I was a generalist.
And most recently was at Balpost, which is a large hedge fund based in Boston, where I was a generalist on the public markets team, doing both public equities and distressed credit.
I think I probably first got bit with the idea of using AI within the investment research process when I was at business school at Stanford.
And this was like 2015 and 2017.
So deep learning was very much in the zeitgeist.
We were still pre-transformer right up until I graduated, but it just seemed inevitable to me at some point that this technology would be really impactful for a lot of the research workflows that I went through and, after spending a lot of time studying and exploring and prototyping, I got the conviction to throw myself full time into this project.
So that's my background leading into starting Portrait.
When you think back to some of those stints, whether it's bow post, slate path, even potentially on the trading floor, can you think of any scenarios where you were spending a significant amount of time that were pain points that are maybe easily solved by AI today?
We'll get into some of the very specific use cases.
But I'm just curious when you look back at that time some of the workflows that you had, where you could see the biggest lift today versus where it was 10 years ago.
It's very different today.
It's interesting because the output of an investment research process is ultimately a decision and hopefully a high quality decision.
It's very unlike other fields where there is actually a widget or a service being provided.
When I thought about the investment research process, there were aspects of it that had friction, but that friction was helpful because it helped me build conviction personally.
I was one of these guys that could never really outsource model building.
I always had to build a model myself in order to feel conviction in recommending a position based on that company.
But there were certainly aspects that I'd say limited me in terms of how productive I could be in finding and researching the best ideas.
Essentially, I would bucket it all into The category of there's just a ton of information out there that I could potentially consume.
Consuming it in a way that was efficient and additive to the research process was challenging, just given hours in a day.
So I would put it in the three main categories.
There's certainly idea generation.
So finding the ideas that best fit my mental models.
I knew there were probably dozens out there that were totally in my sweet spot.
But at any given time, it was very hard to know if I was working on one of those ideas.
There's certainly the initial context building that goes into it.
Especially as a generalist, not even developing a differentiated insight, but just getting the table stakes on how a company and an industry works and the narrative and business breakdowns has always been a very useful resource.
Whenever I'm working on something that where there's an episode, that was always a huge help.
And then I'd say lastly, just monitoring an investment idea.
As a generalist, it was often challenging to stay on top of not just the companies but the surrounding ecosystems, the customers suppliers, competitors.
And if someone lived in a given sector, they usually were able to put together a real-time mosaic, based on all those data points that they're absorbing, about what was happening in the industry and so forth.
And I always found that really challenging as a generalist.
There was more than one earnings season when I got smacked because I was the last person to realize that end market demand was weakening or suppliers were pushing prices through and those types of things.
So ultimately, I would say those are the types of workflows high-level information processing to help triage new data points.
Broadly speaking are the types of things where AI can be really useful.
And we can get into some specific applications of that, but that's how I would think about where AI can fit in a research process without replacing the parts of the process that are important for building conviction.
Yeah, I think that really resonates and seems to align with the people that I've seen getting the most out of it.
A lot of it is productivity and efficiency which, on the surface level, sometimes people can poo-poo.
But when it comes to making decisions and having clarity of thought and being able to take in the most valuable information, I think there's a lot more to it.
So I liked the idea of getting into some of the use cases.
And I actually wanted to start backwards in terms of the buying process and reference what you just brought up there in terms of position monitoring.
What are some of the tangible use cases of AI today that you think investors are using, or you would recommend they use, to just monitor what's in your portfolio and stay on top of it as effectively and efficiently as possible?
I think there are many good solutions today for staying on top of what is happening with an individual name.
There's plenty of ways you can add it to a watch list on any number of services and staying on top of all the new filings and transcripts and news related to that specific company.
What I think historically has been hard but now is much simpler, is being able to cast a wider net of pulling in relevant data points in a much sparser set of data.
So for instance, let's say you own Expedia.
One of the key investment factors for Expedia is always going to be what's happening in the hotel ecosystem and how volume is trending, how pricing is trending, or what OTAs are doing with their distribution strategy and how it relates to OTAs.
The challenge historically is if you're an internet analyst or you're a generalist.
Following Expedia, you probably aren't going to want to subscribe to every piece of news of what Marriott is saying.
You don't care what their profitability necessarily looks like or if they're adding new units to take share from Hilton.
What you really care about is any specific data point which would add to the mosaic around what is happening in the hotel landscape with respect to consumer demand for travel, or if they're seeing trends in market share shift with respect to OTAs.
But historically it was impossible to pick up all those data points unless you really consumed every piece of news.
There was no smart filter on top of all that data.
I think, with AI today and obviously the company I run Portrait, we have solutions that target this.
But even outside of Portrait, there are plenty of ways today to now much more efficiently pick up those data points across that surface area, such that where today a certain level of understanding was unique to someone who just covered travel, you're now able to pick that up much more efficiently as it relates to your investment thesis.
Yeah, I can definitely speak to that as a transport analyst trying to monitor truckers and knowing that there was valuable information coming out of CPG companies talking about freight costs.
I spent many late Thursday nights while the rest of the world was enjoying Manhattan happy hour, control effing through these trucks.
Various transcripts and earnings releases.
So I think that is a major one that has evolved in just the ability to gather all of that information in a much more efficient way and not control effing the heck out of things.
When you next move into the building up a research on a name or the pre-buy research process.
Getting up to speed, this is another one where I think it's the widest net of possibilities.
But when you think about maybe through the lens of how you used to approach it, versus how you're seeing other investors approach it, What are some of the use cases that you're seeing people implement to really get the most out of it and maybe build better conviction or gather more information, whatever it might be?
It's interesting because...
I was someone who, and still do, love doing the pre-buy work.
My wife likes to joke that my favorite Saturday morning activity is reading a 10K.
And the thing that's interesting, what I was mentioning before, where to me that's an important part of the conviction building process is building up that context and feeling like you've read all the material and you can start piecing together the various data points.
Where I think AI can be really helpful in that process is two things.
One is getting you enough information to know whether an idea should be killed.
I used to do this accounting at the end of every year when I was an investor.
How many ideas did I really look at?
A remarkably low number that made it through to the deep research stage.
And a lot of times I could have killed an idea much quicker once I'd surfaced enough information to understand there's maybe an existential risk here.
I'm never going to be able to get past.
Or the management compensation is such, in a way, where I'm just never going to feel comfortable that their incentives are aligned.
And so I think where AI can be really helpful in that process is giving you enough baseline context to be able to triage the idea of oh yeah, this is passing the initial sniff test.
I want to spend more time on it and maybe spend 12 hours reading through all the historical content.
And I'd say the other thing that has been really helpful is just expanding what gets moved up earlier in the pipeline in terms of types of analyses.
So, for instance, I mentioned earlier CEO compensation.
One of the things that I would do if I was really getting into a name is go through the last five proxies and try to map out what are the metrics that the CEO is being comped on and how have those changed and the weighting of those changed.
A lot of times that's a useful signal into what the board is thinking about.
That's a bit of a pain.
And now on portrait, it's a click of a button.
So things that historically would be more in like the deeper dive category, now I can move up.
And that to me is one of the most powerful use cases of AI.
So as an analyst- you're able to turn over far more rocks in a given period of time than you would otherwise, and spend more of your deep research, creative time on ideas that you know have already passed your initial sniff test and are worth that time.
When thinking about really tapping into that, would that be having a list of non-negotiables in terms of whether it's compensation elements or something else that you can screen through quickly?
Or is it?
You don't necessarily know what you're going to hit.
That might break it, but you're just going to get there faster through the various iterations of getting up to speed.
I know it's a little bit technical, but I'm curious mostly if there is a way that you think that you could templatize that versus not.
Oh yeah, there's two categories.
There's definitely things that you can templatize.
So for instance, I mentioned CEO compensation.
There's certain accounting things I would look for with respect to like, aggressive revenue recognition or changes in certain assumptions that would inflate figures.
And then I'd say there were patterns that if I saw that would turn me off.
So one analysis that now is very trivial but took a lot of time historically was I would go back through the last decade three, four years and lay out every piece of guidance that the management team had given, both obviously the hard, specific guidance, but also anything qualitative or soft like oh we expect revenue to accelerate sometime in the second half of the year.
And I think building up a picture of management's guidance style and credibility is really important.
And that's the type of context that someone who's followed a name for a long time intuitively has that being fresh to a name I wouldn't have.
That's a situation where, if my thesis is based on some sort of turnaround execution plan that the management team is undertaking, but I can do the work and say oh actually, over the last three years, every year they've said they expect margins to expand and it never happens.
That would be enough to maybe kill the idea depending on other circumstances.
So I think that's another cool example of it helps.
I think where AI can be really useful is surfacing patterns that historically were pretty painstaking to put together and now can happen far more easily.
Yeah, 100%.
It's a very valid point too on the credibility.
I think there's a trust level.
And once investors show they don't really trust a management team or a stock, it could be a free fall.
And you see that quite frequently.
Yeah.
And it can be really subtle too.
There are times where, if you just look on the Bloomberg screen, it says management beats guidance every quarter.
And then you actually dig into it.
You realize, when they give full year guidance in the Q1 call, they end up revising it down quarter after quarter after quarter.
So by the time they get to Q4, sure, they're going to beat their numbers.
But if you're building your model looking out a year, you probably should shade whatever management's saying at that point subtly.
Yeah, there's effectiveness in just kitchen sinking and taking all the pain in one quarter.
And that can be better than just consistently revising down.
So there's all types of things that, to your point, feel they're definitely quantitative in nature.
But the ability to extract that information was not always easy.
And I think that's an entire category where AI is showing up.
The last section is in that sourcing of new ideas and maybe something you alluded to before, where you have certain frameworks or characteristics of businesses that you really like.
And it's not always easy to find all the companies out there that fit that, or to know if you're looking at a company that might be fitting that.
So where in the idea sourcing, idea generation, are you seeing AI be most effective?
And I will just mention to me personally, it feels like there's always a step before you get to sourcing.
You're usually introduced to a name for other reasons, but I'm curious where you're seeing this coming into play.
There's two main ways I'm seeing investors use AI for this type of use case.
One is understanding companies that are exposed to a given trend or a given development.
So, for instance, when tariffs were first announced, I guess April of last year, there was a lot of investors, especially on Portrait, trying to figure out which companies to be most exposed.
And maybe the second order thinking is even trickier which companies, for instance, have a predominantly us-based supply chain and their competitors have an international supply chain?
Like that type of analysis and ai is really useful for that even out of the box, there's just a lot of world knowledge built into a modern, state-of-the-art model about first and second order effects and who might be affected and so forth.
The more nuanced and, I'd say, difficult case and where we spend a lot of time has been working with firms to find ideas that fit like a nuanced definition of a mental model.
So, for instance, in my past life, I had this mental model of companies that were previously high performing and there were really attractive elements of the business model that you could believe in and underwrite.
And there was some sort of hiccup.
It could be something macro related or really bad product cycle, some sort of execution mess up.
And the market is now revaluing the franchise value of that business.
And a lot of times I could isolate the research to that one specific headwind and figure out if it's temporary or not.
And if it was temporary, then you could buy it.
That's a pretty nuanced approach. thing to find.
It's really qualitative in nature because the headline numbers are going to look bad, whether it's temporary or not, in the near term.
And so finding ideas that fit the mold of that is challenging for two reasons.
One is that just requires a lot of qualitative reasoning along with the figures to do that.
But two, I'd say even enunciating that clearly is hard.
When I ask a lot of investors what are the mental models that you think about when finding a really attractive idea?
Some of them have been very explicit about writing down.
I'm looking for attribute A, B and C, But a lot of times it's more of a gestalt feeling right.
You know when you see it and-
Actually figuring out.
How do you translate that into a query, for lack of a better term, is challenging.
And that's one area where we spend a lot of time at Portrait as well is helping folks look at their history and figuring out like okay, what does a 10 out of 10 idea look like for you, given your historical trading context and firm and all of that.
So I think the combination of those two things is difficult, but when it works, it's incredible.
There are few feelings as exciting in this business as Getting served up a pitch.
Where you're reading it you're like, oh my God yeah, this is really exciting.
Clear my calendar.
I'm going to spend the next week just figuring this out.
That's a really magical moment and something AI has been really helpful with.
Yeah, I can tell you, from doing the show I get to experience all these different businesses and I'm trying to draw the patterns and match them up.
And some of them come more simply than others.
The mission critical part of the supply chain.
That's a small percentage of the overall cost, but they're a dominant player.
They have all the market share pricing power should be there.
That's one that's emerged and I think gained a lot of appreciation over time.
And then there are the more subtle ones, like what you described before, which really articulating them can be more challenging.
But that is an interesting approach and actually something that I've seen.
A few investors who have a differentiated framework try to lean into finding ways to take that approach proprietary knowledge or pattern recognition that they developed over time and get this tool that can get the most out of it in terms of uncovering new opportunities.
I think one of the things that you've made clear to me and other appearances that you made and as we've talked, is there are skill sets, that to get the most out of AI.
And I don't think they're always that obvious, or at least they weren't obvious to me.
But I was hoping we could just talk through some of the things that seem to make a massive difference in terms of whether investors getting the Google search better quality output versus really building on what they're doing, becoming massively more efficient.
And the first is just the prompt writing.
So how would you articulate how important prompt writing is?
Some of the basic things that you think benefit investors from thinking about when going about prompts, and just feel free to wax poetic on it.
It's an interesting question that the answer changes every three months in terms of what makes an effective prompt, because the model capabilities change, as well as the tools available to the models are constantly changing.
In the investment research context and I think this is true for other contexts as well, but particularly research-based workflows with large language models.
The mental model that I think still works really well is imagine you are writing an email to somebody, maybe overseas, who's going to work overnight and is going to be doing a task for you.
And assume they're smart, but maybe lack a lot of context on you.
What information would you want them to have to be able to do a good job?
Then whatever you end up writing is probably a pretty good starting point for an effective prompt.
And to dig a little bit deeper some of the subcomponents.
When I write prompts today, I usually outline a specific task and why the task is happening.
In the same way where, if you give an analyst, build this cost curve, they say okay.
If you say hey, build this cost curve because I think it might be shifting and that could imply something about future changes in the pricing, that's helpful.
So I usually provide some background context.
I provide a task.
Depending on the nature of what I'm looking for, I might specify an output.
I might say, hey, I want your output to look like this.
Have an intro paragraph, then outline these data points.
Show me a table that has X, Y and Z.
That sometimes is useful.
Sometimes that can be hurtful in that you're constraining the model's output the same way of a human analyst.
You might want to give them some slack on the rope, to be able to be creative in how they want to synthesize the information.
And then I usually have a section of my prompts where I have specific task guidelines and then also just general domain knowledge.
So, for instance, in terms of guidelines, these are things that I would just think about.
Hey, maybe I'm doing that guidance prompt.
Make sure you capture soft guidance that doesn't have a specific figure on it.
Because otherwise, if you say just capture all guidance, it's just gonna capture a specific quantitative guidance.
And in terms of transferring domain knowledge.
I have like a set of bullet points that I try to encapsulate.
Here's how I think about being an analyst and some of the things you might learn through your experience.
One of those is I always remind the model like look, you're going to largely be reading commentary from management teams.
They are always biased positively.
It's important you take a skeptical eye to anything they're saying.
And even something that simple really helps because the models have been trained to want to be helpful.
And that usually influences them to be more positive than they otherwise should be.
So just little things like that, where you could imagine taking an eager college student who's excited and believes in the good and trying to impart a little wisdom.
Actually, management teams tend to inflate things a bit.
We'll just wrap that up.
Defining the task, the context behind the task, some specific outputs, if I want it, some guidelines and some kind of domain context.
I think is a really good starting point.
And if you can send that to a human and they read it and understand the task, you're probably going to do pretty well with the model as well.
Yeah, the level of instruction, if you almost humanize it, that was one of the initial things that I think you mentioned months and months ago that made a material change for me.
And then on the idea of context and where we'll use an LLM assume off the shelf LLM for this question.
But when you're thinking about when to just give them massive freedom versus when to load in documents, how does that change the scale and the output quality in your experience?
So I think it's very task specific.
There's a few different dimensions of which it depends upon.
One is, is the nature of the task something that defined and structured and maybe more quantitative.
In which case yeah, it can be a pain if you're using an off-the-shelf solution, but upload the documents.
Because otherwise, generally speaking, an off-the-shelf model is going to have to go crawl the web.
And that's an imperfect, inefficient way to gather a bunch of structured information and you'll most likely get hallucinations, which is why, of course, companies like Portrait exist, because all that stuff's preloaded.
But to the extent that the goal is something more exploratory or creative, it can still be really useful to upload the relevant documents.
But I would explicitly instruct the model.
Hey, I don't really know which way this answer is going to go.
So pull on threads, follow it, chase down leads.
And in that case you can steer the model, especially modern ones that are quite responsive to do a lot of web searching.
Where this matters as well is... how important the task is with respect to accuracy to usefulness.
There are certain tasks where if you're 99% accurate, you're 0% useful.
And that's like self-driving cars where the cost of an error is exceptionally low.
So I think of model building like if I presented a model that had an obvious flaw in it to my boss in any of my past roles.
That would be pretty bad.
I certainly did that on occasion.
Makes two of us.
Yeah.
Maybe you're just doing an industry survey.
What's the history of this industry and how has it evolved to these three players?
If there's a factual error here or there and it's earlier in the research process and it's more of a triage exercise, that's okay.
I'm fine with that.
Having some calibration depending on where the task is in the process, the importance of the accuracy and how structured you want the output to be.
I would vary in experiments.
Yeah, it's quite interesting to see where things matter.
Based on precision versus being directionally right is where all the value comes from.
On your point on experimentation, I'm curious how much this matters.
And this is something that I mentally struggle with sometimes as well.
You mentioned the technology is constantly changing.
So by experimenting or getting better with certain skill sets that eventually the models are going to be able to do on their own.
How much difference do you see in investors or yourself when you experiment with technology, different things in creative ways, when you know that there's a standard decent, quality way that you can solve it already?
I know it's a broad question, just about experimentation as a skill set, but I am curious if you've noticed any patterns with the clients that you work with or you yourself and some of your teammates.
In an evolving technology like this.
I think spending some 15 of your time on experimentation is really important because, unlike past technologies one, the underlying models are changing but two, the capability frontier is evolving as well and is quite jagged.
And there are a lot of undiscovered for lack of better term capabilities that you can pick up on well before others do.
And so having a process by which folks experiment and really spending some time pushing the models can be one helpful in its own right, but two gives you a better intuition of where the models are today and maybe where they're going.
So to add a little specificity to that, I and we at Portrait have plenty of tasks like this.
And even me personally, I have some tasks where I know the models can't do this today.
Or maybe they can do a small percentage of the time.
But anytime there's a new model that comes out, I run my suite of 10 different exercises across it.
And I, one, can get a feel for how much of a leap this model is.
But two, I might now need to update where I think the edge of that capability frontier is.
And a lot of times, once you start doing that process, it kind of builds on itself where...
You might have an experiment.
Maybe it is building a cost curve and how that's shifted.
That's a type of task that can take a long time and be challenging.
And now the model can start doing it.
Well, great.
Now you start having ideas of well, I wonder if I layer in a long-term cost curve versus a marginal cost curve.
And there are different things you can start coming up with for ways you could actually apply it within the research process today.
So I would say for anybody certainly companies do this, but individual investors have a suite of 10 different things where if the models could do, it'd be great.
And just constantly run these against the model.
And once they are unable to do it, save that template and reuse it as part of your research process when working on a name.
So I think those are some of the things that where experimentation can pay a lot of dividends.
I would bucket this in experimentation.
But one of the things that you laid out when you talked to Brett from Fundamental Edge was just writing prompts in different forms.
So start out with the most simple prompt that you know is probably going to give you a very broad, mediocre answer.
Then take it layer down in more detail and specify more detail and then see how the responses change based on how you're instructing it via prompts.
I did it a bit and it made me appreciate how it's sensitive to maybe, the ordering of questions, or if I specify the ordering of questions and what I'm loading into the context window.
And it doesn't just impact that single exercise.
It makes you appreciate this is a technology that really doesn't have many buttons that tell you oh, this is four wheel drive that I use in this condition.
It is just a chat and you have to play around with it yourself.
And just on that, I think one thing that's really cool I mentioned earlier with the way I write prompts is I think about sending an email to someone overseas.
The difference with an LLM is that the response is, for all intents and purposes, instant.
The real insight, I think, comes from doing exactly what you just described, which is iterate on this.
The cost of sending a single query is trivial.
Start simple, start adding complexity as it's helpful.
It's really like a two-way dance to end up in a spot where the AI really is acting like a useful analyst and You don't need to wait overnight to get the content to then see how I get feedback.
You can give that feedback real time and adjust things.
And that becomes super, super powerful.
Your prompts are loaded into portrait for their primers or whatnot.
And you see the layer, the levels of detail that go into those.
And it's actually quite helpful.
It made me feel a little down on myself in terms of what I was prompting prior to seeing those.
But I think, yeah, it definitely ranged true.
Taking it up a level.
I have seen quite a few individuals who are getting a lot of value out of AI, but it almost feels like it's siloed.
You could have two individuals at the same fund that use entirely different tools, and they're getting value.
Obviously, the enterprise or the fund is benefiting from all of it, but it doesn't necessarily feel strategically aligned or there's too much collaboration in regards to it.
Have you noticed anything in terms of best practices for funds, just to get some type of adoption that's aligned or collaborative?
Because investing is ultimately a lot of the research is building conviction and people build that in different ways.
It's challenging to mandate changes in a research process, because if I was at my last job and the head of the firm said OK, now you must use only sell side models for your models, I would really struggle.
I can certainly sympathize with.
I know a lot of firms have been trying top down to say hey, we don't want to fall behind on AI.
You need to start using such and such tool.
And I think that can sometimes work in certain ways, but in other ways it can backfire and people push back.
What I've seen the most adoption and firms that are taking the most advantage of this.
I'd say it's finding the right balance between having firm-wide initiatives that don't force people to change their behavior, while letting individuals experiment and figure out where AI is going to be additive, and reducing friction without negatively impacting conviction.
So, for instance, just using Portia as an example, there are firms where, at the firm level, we spend a lot of time building idea generation systems, screens that are super bespoke to them and their process.
And we just run those like an outsourced analyst.
And that's really value add because all we're doing is not asking anyone to change the process.
They just get a bunch of really interesting pitches that now they can spend time on if they want.
And similarly, the thesis monitoring has become a really big thing.
And the nice thing about thesis monitoring is it doesn't require a change in process.
All we're doing is we take a ticker and maybe your investment thesis and then we're just pulling in data points that at least our system believes could be really helpful and incremental to your thinking.
And again, that doesn't require anything new.
It just requires looking at a new insight when it arrives and ignoring it if it's not relevant.
Whereas I think for the actual day-to-day asking queries, building outputs, it really just depends on the user.
And obviously working with a firm that specializes in this and has built software specifically for this vertical can be really helpful for driving adoption because, unlike a general tool like ChatGPT, someone building software in their space can customize the UX around the specific workflows.
But ultimately I think that adoption has to happen bottoms up and people need to feel comfortable with it to be able to continue making high quality investment decisions.
And to the extent people have changed their process and many people have they've done so because they feel trust in it.
And I think building that trust has to happen on the individual level.
Yeah, it's actually an interesting perspective that I somehow hadn't thought about.
But when it comes to any type of investment, research and investment process, people do it all differently.
Within the same fund, they might have the same core things that they look for, but the path to get there is quite different.
And even model formats...
I can remember many of arguments over having a single specified template and people moving off of that.
So yeah, I think there's a lot to that.
And maybe that opened my eyes a little bit.
On the practices that firms or individuals can do in terms of investing in things that maybe aren't obvious in value today, but as the technology evolves, maybe they'll be able to leverage more.
And I mentioned...
Before we talked about, there's the famous example of the sports analytics teams that had been tracking data well before sports analytics were a thing.
And they were only able to really benefit from all those data practices decades into the future.
But the teams that never kept up with it, it's really hard to go back in time.
So are there exercises or practices that you think funds or individuals can start to implement that will pay off in time?
Obviously, everything we spoke about earlier with experimentation certainly applies here.
But I think one thing that is going to be really important and already was historically, but I think carries a lot more weight going forward is just the importance of documentation, of thinking and decisions and research.
Essentially these models, the usefulness of them rises exponentially with the amount of context they're given.
As the model context length expands and as their agentic reasoning expands, gets more heavily utilized to do longer running tasks and they ultimately shift from being a helpful research tool to something that lives within a firm and can execute a research process.
Their ability to do it is going to be a function of how much data they have on how you and your firm operates.
So I think firms where they have documented memos and even write up short paragraphs on why they made a trading decision.
It's hard to know ex ante which data is going to be used and how.
But I think it's a pretty reasonable bet that having that data at a minimum is helpful for the humans but will certainly be helpful for the machines.
Because I think a moment ago, we were speaking about adoption.
Challenging is... loading in that context.
It's the same challenge that goes when you hire a junior analyst needing to teach them everything about how you think and your past trading history with a given name, and all that information just takes a ton of time to convey.
And so, to the extent that that data can be captured in real time, that forms an enormously valuable source of IP that will make AI unique to you.
And again, it's a little hard to know upfront how that's going to be used.
And it probably takes some creativity to figure it out, but hard to imagine two or three years from now, that every piece of data isn't being used within a model that is operating within the context of a fund.
It's a perfect example.
I could speak to on ramping at places where all they had were the memos which were the buy and sell decisions for positions which are valuable in some ways, but they're usually overly polished versus those that, after earnings, they would have their quick blurb.
Maybe Amazon entered the space and it was something to monitor.
And then a month later they had a meeting with the management team and it made them feel a little worse off.
And then another quarter comes out and it's clear Amazon's impacting them.
And the memo might just say Amazon's movement, but to see the thought process and how the team makes decisions.
There was something very valuable in me getting up to speed.
Again, I think it goes back to that lens of almost thinking of it like a team member, in the same way that you would give them the most ammo to align with how you work and what you do.
Going back to some of the quick hitting questions, thinking about down the road and where we're going.
I already brought up context windows.
We'll use LLMs again, just thinking about some of the most used tools.
When you think about where investors or where you personally find the context window advantage has the most impact, or how you adjust for the challenge that I think many investors face, which is I want to get as much information in there as possible, but sometimes I'm limited by it.
How do you approach that?
And how much do you think you should worry about that as an investor?
It's super dependent on what tool you're using and in what context you're using it and the task.
To go a little deeper on that, the context windows obviously have been expanding, although really in the past year they've been flat, with Gemini at around a million, GPT I think around 400000 and Opus or Cloud around 200.
Where the models have gotten more capable is...
Using that context.
Well, I think historically models struggled if you loaded in I don't know the last five 10Ks and let's say a 10K is like 100000 tokens, so 500000.
The models historically would be really good at, say, if you ask, say, what was SG&A in 2023?
They call that a needle in a haystack problem.
They would struggle if you said, build a three-statement model using these.
The model needs to apply its attention across lots of different pieces of the context window simultaneously.
I think that still is a roughly helpful rule of thumb, which is the models are generally.
If you're using a ton of context, you need to do better if you're loading that in, because you're looking for very specific data points.
That being said, I think a change versus if we had this conversation three months ago is models have gotten very good at using the context intelligently.
If you're thinking about building a project in portrait or notebook, LM or something like that, I wouldn't really hold back.
I would give it pretty much the entire corpus.
And you're seeing this a lot in software engineering, where cloud code and other agentic coding tools are very capably exploring super long context in a way that is efficient and focused on the task at hand.
So in the same way, I wouldn't give a junior analyst a task and say these are the only four documents you can use.
You'd want them to ultimately have the agency to choose what content they're going to use.
And I think that holds true.
All that being said, if you're going to use the raw model itself, you do not want to overload the context window and use 70 of the context window.
You will see a degradation in any complex task.
In that case, you should break it down.
But hopefully the much easier way to do this is use a tool that manages that complexity already.
So you don't need to worry about.
Okay, what percent of the context window am I utilizing, and so forth is how I think about that today.
On memory, I think it's obvious where as memory has now been implemented in a lot of these tools, it customizes you and that iteration, again, I'm going to overuse the lens of the employee overseas that you're creating a better, more aligned working environment.
Are there any non-obvious impacts that you think memory will have?
I think the interesting thing about memory is today, as it's implemented, it's far worse than a human's memory.
It is super limited.
It's expensive in the sense of having deep memories takes up a lot of context.
It lacks a lot of the nuance of our memories, are really useful both for specific facts, but more so for the abstract concepts that they create and that we learn.
When you think of those memories, like a lot of times, that connection is where they become really powerful and that's become pattern recognition for investments.
So in the near term, I think memories is a useful tool for all the obvious reasons that you can think about in terms of just adding more context.
That persists in the context window.
Longer term, everyone knows all the fallibility of human memory.
And this is a little bit of my pay grade in terms of the actual ML research and how memory is being incorporated in different ways.
But you could imagine any one of the architectures that are being experimented today with memory and having short-term memory and longer-term memory and the ability to look up memories, if implemented correctly, both in terms of how the models are trained as well as the hardnesses around them.
Imagine having a model that has lived every single one of a firm's investments and maybe things they even passed on.
There's a world where it becomes so much more powerful than any given human.
I look at some of the investors I've worked for and they just have this insanely, eerily intuitive judgment about things.
A lot of that is not just memory, specific facts, but memory that comes from the experience and scar tissue.
There's a world where that certainly could exist in models.
And I think in some sense people talk about AGI being the ability to be generally intelligent, intelligent.
I think a really interesting test for a model of this AGI is can it predict the future?
You need a really good world model and understanding of lots of variables to predict the future.
And that's what an investor does.
And memory is clearly a really important ingredient to that.
So I'd say in your term, I'm a little bearish on just how big of an impact it's going to be relative to just copy pasting that context into a window.
But longer term.
I think that's the unlock, where this goes from being the useful tool to the real core driver of a lot of the researcher firm is doing.
Well, I'll separate near term.
I'm almost taking away that it might be more valuable to limit the memory, therefore reducing the context window and investing more into the prompt that I'm using.
I know they're not like for like, but ultimately memory is customizing to you.
And if you could just put that in the prompt, that might be more beneficial than sucking on memory.
It's a near-term convenience.
So, in other words, if there's knowledge you want to impart in every single time you interact with the model yeah, that should live in memory.
Or if there are aspects of working on a name and you've maybe done 20 queries on a particular name.
You maybe want some sense of the memory in living in the model.
So if you reference something back or it already knows it's did XYZ analysis, it can do that.
I think that's where it's useful.
It's a bit of a shortcut for just loading in context that you don't have to repeat.
I definitely would recommend spending a lot of time being thoughtful around to the extent you can manage the system prompt and off the shelf tool or anything like that.
I think we're just very early innings from a technical perspective, tapping what it means to really utilize memory in a thoughtful way.
And just to illustrate that, and these models have such incredible world knowledge by essentially reading the entire internet in their pre-training.
So they clearly have the capability to store in their weight a lot of knowledge and then learn important concepts from that knowledge.
That leads to things like reasoning.
So it's just a matter of time before that same paradigm applies to continual learning on the job.
And that, to me, is going to be really, really fun.
Yeah, I do think about if Bridgewater has really been recording all those meetings for all of this time, what all of that, that input, could do eventually and what could happen with that technology.
I don't even have an idea of what it could be, but it just feels pretty unique.
And seeing how people are testing this, even with their own personal recordings and creating their own things, thing I wanted to talk about because it gets brought up.
It's a buzzword, it has very practical applications, but agentic AI the definition gets a little bit fuzzy sometimes.
But just thinking about agentic AI as a topic and as it applies to investors.
If you could just talk about where that's going, how you would even categorize that within the various tools and anything else you would mention on that.
So there's lots of definitions of agentic AI.
I think of it essentially, as the model has the capability to reason reflect, take action, reflect on those actions, ultimately in the pursuit of a goal.
And there's obviously different levels of scaffolding and constriction around what the model can and can't do.
What's really exciting, especially now, is agents have started to work.
And interestingly, when we first started Portrait, we tried and we did actually have an agent that was running using GPT-4.
And it was so hard to get the work.
And the way you got it to work was you had to be so prescriptive about what the model could and couldn't do.
I think our system prompt was 30,000 tokens or something crazy.
Really had to limit it because at any given time the model, if it started going down the wrong direction, it didn't have the ability to self-correct and be thoughtful about what it's learned and how it should adjust its plan.
And that required just a ton of prompting.
What's cool today and I think what was happening in software engineering is the leading edge of what's going to eventually happen in investment, research and really any other knowledge field is the models have now become smart enough to to do longer running tasks through using tools and updating its thinking process as it goes.
And you're seeing this with things like Cloud Code and Codex, where people are leaving these models alone for multiple hours and it is dynamically operating like a software engineer to understand a code base, make changes to it, test things out, see an error, go look at the trace and figure out why there's a problem.
Make a fix.
That type of thinking is certainly really challenging and that now works.
The nice thing about code is it's the perfect environment for these models to figure this type of capability out, because one it's all just text files at the end of the day.
It's all contained usually within a repo.
So all the context is locally available and And it's super verifiable.
The code either runs or it doesn't.
Investment research is obviously much different.
None of those things are true.
The context is in many different forms and very broad, and some is more accessible than others.
It's also much more qualitative in terms of output.
But the form factor of can these models reason iteratively and arrive at complex answers that require on-the-go changes of plans?
The answer is yes.
And it's just a matter of the engineering work to get those models into the right context so that they can operate like a junior analyst and ultimately like a senior analyst doing long running research.
It's definitely still a buzzword, but there's a lot more meat behind that buzzword than there was maybe a year ago.
The point on code and I often bring up chess, these constrained environments that are still very complex, but they're constrained by certain things.
You tend to see more of a clean iteration cycle and more productivity that comes out of it.
And the more complex and adaptive the system, the harder it is to control the evolution.
And so it's one that I think will be interesting to see how it all evolves.
But This has been fascinating.
Thank you for going on some more philosophical tangents in addition to the very tangible use cases.
It's been a pleasure.
So thank you for sharing the knowledge with us.
Thank you.
It's been really fun.
To find more episodes of Breakdowns, ranging from Costco to Visa to Moderna, or to sign up for our weekly summary, check out joincolossuscom.
That's J-O-I-N-C-O-L-O-S-S-U-S dot com.
