I think how the actual gets to solve these problems may be even more useful. Is it a new architecture? Or is it just 10 times more training?


There are other aspects of doing mathematics, forming conjectures, making connections, articulating general principles. Mathematics is not a single, unified, one-dimensional activity. But I think it's a good benchmark because it's definitely beyond our current capability. So any progress for that goal will be useful.


Yeah, I guess a lot of our focus is on taking known problems and solving them. I'm curious how much of mathematics in your eyes is the problem of coming up with the right new problems and forming the right kind of aesthetic conjectures.


Yeah, there are so many types of ways to do mathematics. I mean, sometimes taking something that's already been, I think, understood, but explaining it in a much better way. Unifying two concepts into a single more abstract concept without actually changing much the arguments, but just changing the perspective or the emphasis even. Sometimes even giving things the right name can be extremely influential. So the concept of what a number is has been generalized over and over again in mathematics. It's almost unrecognizable to say what the Greeks would think of as a number. But these are really important conceptual developments.


There's a social aspect to mathematics. I mean, it's not just... Nowadays, if you want to interest people in your work, you have to actually do a good job communicating it to other humans. And the effort of doing that has actually transformed sort of maybe the directions in math we go to. And I mean, some areas are more popular than others, but also we try to arrange, we provide motivation for what we do a lot more than we used to in the past. Partly because we also have funding agencies that we need to make money from and so forth too. But there is also, I mean, it's not just individual mathematicians solving problems. There is kind of a super intelligence of the whole community trying to move in certain directions.


Yeah. That part of... So there's a social aspect, which maybe the AI the automation won't touch until the AI has become fully-fledged members of the community. That's socializing with us. That's way in the future.


Yeah. I mean, it's so multidimensional. I mean, I think you can't reduce it to one activity or another.


Natalie, just a very quick check on time.


I know you've been working on this for a long time, cool.


Natalie, just a very quick check on time. We have three minutes, so maybe one more question.


Fantastic. Awesome. Yeah. I'll end with maybe one last question. So one of the primary goals at OpenAI is not just to build AGI, but to build safe AGI. And we imagine AGI eventually becoming very powerful with the ability to do its own research, recursively improve itself, think about really hard problems and maybe interact or act in the world. And I'm curious how much thought you've given to the AI alignment problem, making sure the AI doesn't go rogue or have any kind of large negative societal impact.


Yeah. I mean, I think a lot of it is... I mean, it is an interesting question, but the other technologies that we have, the existing automation we have, we almost never shoot for 100% automation because precisely because of all of these risks.


So, I mean, we can partially automate driving and aviation and financial trading and so forth. But we don't completely run things on autopilot and we shouldn't, I think. I mean, there's a diminishing returns to automation up to a certain point. I mean, so I think 100% automation is not necessarily the end goal. I mean, maybe just automating the humans, but not replacing a human.


I think there'll be so many visible examples of 100% automated AI going amok. I mean, not in the sense of killing all humanity, but just sort of failing in hilariously bad ways that I don't see... You don't see people experiment with completely automated cars. Well, I mean, we're trying a little bit now, but I mean, I think at least in the near term, in the next few decades, we can already leave a substantial portion of this AI just by partial automation and just using AI to empower humans. And theoretically, you could turn everything to a pure AI, but it's not clear that outside of some very specialized subtasks that this is necessarily a thing to do in general. In part because we can't solve the alignment problem, but maybe we don't have to, to have a much more useful technological tool.


Yeah. Thank you.


Thank you, Greg. Anybody else want to jump in? Oh, Yufi, I see your hand. Welcome. So happy to have you, Yufi. I think you know some of the panelists.


Thank you very much.


Hi Yifei, good to see you again. So as, you know, I'm a research mathematician myself and one of the often frustrating routine challenges is reading and checking whether some written proof is correct, right? So this includes my own writing draft as well as other people's writing on the archive. I think of both refereeing, but also just trying to understand what's going on in these proofs.


And I wonder, you know, in practical terms, how soon can we expect an AI tool that could dramatically assist us in this part of the research process?


It's already beginning to happen. I mean, you can already upload a PDF, maybe not directly to GPT, but there are other APIs and things that do this. And, you know, so I use something called ChatPDF. I'm not quite sure what it's powered by. And it can give kind of a high level description of the thing that they can tell you like what the main theorems are. It can summarize in its own words what the main ideas are. It's not completely accurate, but it's a start.


And then at the low level, we have these formal proof assistants that can take individual steps and turn them into rigorous formalized proofs. And there's beginning to be software that they could turn those formal proofs into a very interactive document that you can use to explain. And at some point, like maybe both the high levels of AI interpretation of a paper and the low level formalization will start merging. Once formal proof verification becomes user friendly enough, hopefully the AI, the language models provide enough kind of a pleasant interface that the average mathematician can write their papers no longer by typing LaTeX, which is normally just like by explaining, you know, in plain English to an LLM, which will then write in LaTeX and in a formal language. And the end product could be something that is already in a format suitable for explaining at whatever level you like.


I mean, GPT is already great at taking a topic that is well covered. I don't know, say nuclear fusion. And if they explain it at the level of a 12 year old, as an undergraduate and so forth, it's already good at doing that. You can't do that yet with a math paper, but that's well within the possibility in a few years.


Have you tried feeding one of your own proof lemma drafts into GPT for the check for mistakes?


Not to check for mistakes. I don't think checking for correctness is the strength of a language model. I haven't really tried.


Yeah. Some very bite-sized statements, I've tried it. And then, yeah. And then to get the high level picture, it's somewhat accurate. It's not really ready for prime time yet, but it has potential.


Daniel. I just wanted to add a little bit to the possible promise of formal math for this exact problem. So Peter Schultze was genuinely in doubt of one of his results. And with the lean community's help, they formalized it and gave a lot more confidence in the veracity. My understanding is that most of the work was after the formal statement, and something that there's no philosophical barrier to automating with a more advanced GPT. And so really, there's maybe just a little bit of a connection there of also getting the language model to write the formal statement in a way that the human can audit, who is maybe not an expert. There's a clear path to automatically verifying much more sophisticated. It could be that the only bottleneck is data. Like if we have enough, if we have more data sets of people formalizing proofs, maybe the language model can just take it from there. Thank you, Daniel.


Anybody else wanna throw their question in the ring?


I've got one.


Nikhil, what's on your mind?


Hi, Nikhil. I had a lot of people... Hi, Terry. Hi, Akub. So it's more a question for the AI researchers. So there are a lot of questions of the type, when do you think these models will reach graduate level or postdoc level or so on? But how do you begin to answer these questions at a scientific level? Is it based on looking at some scaling laws or like how do you actually begin to make these predictions?


Yeah, this is a question we've thought a lot about. One thing we can predict with a good degree of confidence and what scaling laws are about is that there is room for these models to continue improving and we can predict how they will improve at the core task we are training them on, which is predicting the next word in arbitrary text. And I think it's a good question. And so for that, we have concrete predictions. We can kind of say, if we expand this much compute in this way, we'll get a model that is this accurate. And the big question is, how does that correspond to actually being able to do things and actually being useful? And on that, our understanding is much more limited. And I think for GPT-2 and GPT-3 and GPT-3.5, every time you see these models, you see like, well, they are clearly a big step change from the previous thing, but probably nearing the limit and probably the next thing will not be quite as good. And probably we are missing something fundamental to really push it forward. And I think so far we've been surprised every time with the new capabilities that these models show. So I think maybe this time expectations are maybe a bit higher for the future and we are less confident there is some clear plateau that we're approaching.


I still think we can be more, we can still develop methodology to actually be able to predict when we will be able to solve particular problems. So one thing that we have studied for GPT-4 is trying to predict when we'll be able to solve problems of a certain difficulty. Consistently, and we've observed that there is also a certain scaling law that if you can solve a problem very infrequently at some scale, at some level of being able to predict the next word, then as that improves, you can predict you will solve problems much more consistently later. And there are ideas on how you could extend that methodology to problems that current models are just completely unable to solve, but that's an active research problem.


Now there, I want to add another very quick comment here.


To the already, to the comments that were said earlier. Some of the remarks describing the capabilities of the models were like, well, this is an undergrad and then you have a grad student. And one thing that's worth keeping in mind that these models are extremely non-human in their capability profile. It's like a human being some without frontal lobes or it's something different because it's obviously very superhuman in some ways and it's very not human level at all in other ways. So it's radically better than GPT-4 is radically better than an undergrad in terms of its breadth of knowledge and the way it sees connections and the way it has this obviously superhuman intuition. But on the other hand, if you botch simple calculations and make simple errors and fail to find obvious flaws in a proof, be unreliable. As the reliability crosses a threshold, that's where things will stop.


start to get really different from what they are right now.


Thank you, Ilya. Thank you, Nikhil. Wonderful question.


Ryan Kinnear: Hey, so thanks for taking the question. So there's a lot of discussion about how you might use language models to do math, but not so much discussion about the math of the language models. So maybe is there, so for an example, there's a Millennium Prize problem inspired by fluid dynamics. There's certainly lots of other interesting pure mathematical ideas that have come out of other areas, engineering or so forth. Is there anything related to language modeling or machine learning that you think has inspired a truly interesting purely mathematical problem? I've asked some experts in this. So far, I don't, there's nothing on the level of say, P goes NP, which has really motivated a lot of very good mathematics. I think machine learning is still quite an empirical science in many ways. And it's a bit opaque, actually quite a bit opaque how these things work, even though a basic theory is there. So I think at the current stage of knowledge, not really, but people are still trying. There are some non-trivial math questions coming out of, inspired by neural networks, but nothing as sort of as central as say, P equals NP. What I found talking to applied mathematicians, a little bit of math can go a long way. I think the way one of my colleagues described it, it's like chapter two of a math textbook is very useful for applied mathematicians. But then after that, it's decreased exponentially how much value they get out of the remaining chapters.


Scott Aronson: Do you have a question for the panel?


Hi Scott. Yeah, hi Terry. So when we think about the greatest open problems of mathematics, like P versus NP or Navier-Stokes that were just mentioned, I can imagine someone saying, well, maybe we shouldn't kill ourselves at this point to solve these things because in 10 years or 20 years, it's as likely as not that AI will be able to help us. Maybe that's faster than humans will be able to do these things. But I could imagine someone else saying, well, for the sake of human dignity, I want humans to have solved these things. And if we only have a 10 year deadline for that, then we better get cracking now. I mean, do either of those ways of thinking have purchase on you?


Well, the great thing about math is that the problems are not fixed. I mean, we select our own problems. And this is something that sort of already happens. Like mathematicians get scooped by other mathematicians all the time. And it can be emotionally annoying when it happens. But it's also, but actually it's a good thing. I mean, so once you get past the initial psychological shock of someone proving what you wanted to prove, well, first of all, it's validation that what you're working on is an interesting problem. But also often the solution to problems opens up many more problems that you can now solve. And it's a net positive. So I mean, it's not, mathematical problems are not really a finite resource. So it's certainly not the constraining, limiting risk. The bottlenecks to mathematics is a lack of problems. So I would not worry about AI causing a math problem shortage. That's really, out of all the AI risks in the world, that's like number 110, really way on the bottom.


Well, until it can solve all the problems, right?


Yeah, but then the fact that it can solve as a subset of all the math problems is not, that will be a very niche concern, I would say.


Daniel: Did you want to hop in? You're muted, though.


Yes, hi, sorry. To the point that mathematics does not have a finite number of problems, but has a potentially infinite number, one of the challenges to scaling language models further is the limited data from the past. And the idea of there being this infinite set of, this infinite playground, where, especially in a formal regime where you can automatically tell whether something is logically sound, it's very tantalizing from a research perspective. Do you have thoughts on how you might formulate an objective to say whether a formal theorem is interesting? Or do you think there's any way to turn this into a big open-ended game that language models can play?


That is actually a great question. Yeah, so Go was mentioned, AlphaGo was mentioned previously. One thing that language models are potentially very good at is making qualitative assessments of whether, for example, whether a mathematical statement looks easy or difficult, but also whether it looks interesting or hard. And maybe once we have good AI tools of doing that, that can help us guide mathematics in very productive ways. So it can actually help us sort of generate the vision for future direction. But there's no data on this. There's no data set of what is interesting mathematics and what is not interesting mathematics. So correctness is a much more promising near-term goal, because we at least have a lot of data on what states are correct or not. But yeah, in the far future, I can see AI actually making really good value judgments, not just in mathematics, but elsewhere. But I think we do need some non-data-based AI tools. So language models are great. But yeah, they need data, huge amounts of data to function. And so a large part of doing science and mathematics is actually moving into regimes where we don't have good data sets. But I mean, I don't think that's going to happen. But I mean, maybe still a way of reasoning is so similar enough to previous years' reasoning that you can use reasoning as the data. I don't know. I mean, humans don't reason by massive amounts of data exclusively. I mean, they definitely do to some extent. It's not the only way we think.


Quick follow-up. You mentioned before about using GPT for a somewhat routine idea generation. But have you explored its sense of taste in conjectural definitions?


I mean, the time that I haven't tried very hard. I mean, the thing is, it captures the form of these things much better than the substance. So I've asked it to generate a potential talk, like someone's playing with the Riemann hypothesis. So what would the title and abstract look like? And it will produce a convincing looking sort of talk that I would actually go to. But then you ask it to elaborate, like what if you said the talk, and then you realize it's all nonsense. So I mean...


I mean, it's more like the question of, is this theorem statement, is this conjecture interesting? Which is a little bit more of an intuition. I haven't tried, but I suspect that it's just too far from the data sets that it's trained on to. I would expect that it would not give great results, but I haven't tried.


On the comparison of math to Go, I think one important feature of Go and other games that deep learning has found success in is that the space is not only kind of has a closed infinite number of problems, but also it's kind of smooth where you can improve a bit on easy games or easy positions and go from that and improve beyond human ability. And in comparison, naively at least, the space of math problems feels much sparser where you have to make big leaps. Okay. It depends on how you sample. So if you randomly generate a math problem out of random symbols, then neither humans nor AI will ever...


Inverting a cryptographic hash function is basically impossible. That's why we use cryptographic hashes. AI is not going to break crypto. Crypto is done correctly. It can be refined, the loopholes and so forth. But yeah, but the, you know, it's like the drunk person looking for the keys under the flashlight because that's where they can actually have a chance of solving the problem. You know, the problems mathematicians solve, the areas that we work in by nature are the ones where the solution space does have structure. And I mean, the structure often took us decades to realize, you know, that there are certain directions in which you can move. And, you know, it takes years for humans to understand how to work in advanced geometry or whatever. But AI can navigate the space in principle quite well.


Yeah, so I mean, yeah, I mean, if you focus on the areas of math where mathematicians actually like to work in, I think the situation is a lot better than in just random generic math problems. But I'm curious, even within those spaces, right? It's not clear that you can find a smooth gradation of problem difficulty, let's say, as that of, you know, ELOs in Go opponents.


Yeah, maybe not for mathematics in general, but for very localized tasks, you know, like you want to prove a specific theorem, you know, using a certain set of tools. So the type of thing that you do all the time in the form of proof verification. I think AIs could have a reasonable chance of assessing what intermediate steps might be sort of halfway between your hypotheses and your goal, and how to split up a problem into two problems of half the difficulty. And then have sort of tree branching sort of AlphaGo style, maybe way of solving sort of specific lemmas and things like that. I think that has a game-like feature to it that might actually be feasible.


Thank you. Shrenik, Shah, I'd love to hear your question. You'll just have to unmute yourself. The icon is at the bottom left center of the screen, the microphone icon, Shrenik. The very bottom of the screen below Daniel, Ilya, and your face. You can also type it and I'll ask it for you. I think earlier Shrenik had typed something he asked, perhaps I'd ask whether AI might be in the future more effective in fields that are theory building, like the Lang Lang's program versus technique development, like combinatorics, some kinds of analytic number theory.


It's not really quite, it's not a really well posed question because that's a fake dichotomy. Yeah, so I mean, that's much more in the purpose of me as a mathematician myself than the theory building. It's theoretically possible. I think, again, there may be a lack of data. So we have a lot more data on, there are millions of math problems out there, ranging from elementary level to advanced graduate. There are not millions of math theories out there. So it's not clear to me, that is maybe I mean, again, you could maybe generate plausible pseudo math theories, things that look kind of like math theories, but then they don't have the detailed inspection makes them fall apart.


So this actually is something you were just addressing in a way, but suppose we get to a point in the future where AI has the ability to solve some kinds of math problems and not others. Do you think it's more likely to be successful at addressing the types of mathematics where you kind of have to build up a large amount of structure, create new mathematical notions and objects and so on to build up to proving some major conjectures? Like for instance, like the proofs of the various conjectures about pure and mixed motives or the resolution of the Langlands program, their balance and conjectures versus areas which feel like there's a lot of different techniques that are developed. And then when you combine them in extremely complex ways, you can prove stronger theorems like mixing sieve methods, like sub-convexity bounds and things in analytic number theories, like the example I have in mind.


I think, I mean, it may not necessarily be language models, but I think there will be progress in both these directions made by different AI tools. So, you know, different fields have different bottlenecks. So like some, as you say, we know what the tools are, but the question is how do you combine them to optimum effect? There are some where we don't have the right definitions or the right conjectures to make. And yes, some we don't even know what the questions are. Like the questions that you start, like the initiative that you would make end up eventually becoming less relevant as time goes by. So in all these areas, there is a chance that AI tools could help. I mean, it's a question of what architecture you're using, how they interact with humans, what data sets you have. Yeah, I don't know. But as I said before, also, I think the way we will do mathematics, it's, these questions may be moot in 20 years because just our entire practice of mathematics is just, it's unrecognizable from the way we do today. And so arguing how, you know, as I said, like in the early 20th century, like a lot of mathematicians were computing integrals, exact integrals, and doing a very basic numerical analysis. And we just don't do that stuff anymore. And so arguing which automated tool solves these problems, any automated tool sort of solves those problems, but that's not what we do. So I think it's premature to ask the question.


Thank you, Srenik. Well, it's our last 10 minutes, and I fear we might not have Terry back for a very long time. So does anybody else want to throw their question in the ring? We are happy to give everybody their time back, but again, this has been such a special moment. And if anybody's holding back just because you're shy, please don't. I think there's someone in chat.


Yeah, I see. Oh, Ilya. And after Ilya, Ryan.


Ilya, you can go ahead and unmute yourself.


Oh, I apologize, I don't have a question.


Okay, okay, how about Annie Kwan? We haven't heard from Annie yet.


Hi, nice to meet you. Thank you so much for this session. My question is related to just if AI is going to take over a lot of these, you know, basic math. I'm just curious how we can educate the next generation of students, like what we should focus on given that we have AI now, which we've never had before back when we were students.


Yeah, well, you know, we've had to adapt in the past to, you know, calculators and the internet and the question and answer side that can already, like you can crowdsource a lot of homework questions to humans already, and now you can crowdsource a lot to AI. I think, you know, AI tutors will become very widespread. So, I mean, that, you know, that basically you have 24-7 office hours where you can talk to a teaching assistant, that AI assistant that will, you know, it's often in a math problem, if you're not supervised, you can get stuck at a step and there's nobody to nudge you. You can just, it can be very frustrating. I think we will have to be more creative about how we design assessments and how we have to somehow weave in the AI tools into our homework that we assign. As I said, I gave some examples of trying to get students to critique an AI-driven answer or to, I don't know, maybe use the AI to generate a new question and try to challenge each other to solve it. I don't know, it's very, very early days of how people are going to use it. As I said, textbooks, I think, are going to change quite a bit. Textbook technology has not advanced that much in hundreds of years. I mean, the graphics have become slightly better in typesetting, but I think we could really see really interactive textbooks. And, yeah, and one way you can calibrate, you know, if there's a passage that's too hard, just say, can you explain this part of the text at the level of a five-year-old, or can you make it into an animated comedy sketch or something? And, you know, everyone's got their own different way of learning, and like really personalized, mass personalized learning could really be a potential thing that AI can deliver.


Annie, thanks for your question, it's good to see you.


Thank you.


Stephen, did you want to unmute yourself and ask your question? You've been writing a lot in the chat.


Sure. Hi, Terrence, yeah, thank you for being here. I wonder how often you've been pitched on working on alignment directly, and in particular, you know, is it off-putting, what has been interesting? There's obviously a lot of discussion of getting very smart talent to people like yourself interested in working on it.


I did get a phone call from Sam at one point. Yeah, no, it's an interesting question. I think it's not my comparative advantage. I mean, it's not a question that is really mathematically well-defined to the point where sort of the skills that I have are really the right ones. It may be that you really need people from the humanities, actually, to address this problem currently. Yeah, no, it's a great question, but there's plenty of other people working on it, so I'm happy to let them think of these hard problems.


Daniel, did you want to hop in? You've had your hand up for a while. But you're still on mute.


Hi, sorry about that. I was going to ask, Terry, why are humans so good at math in the first place? It seems like phenomenally interesting that these abilities are so strong, despite it not being that obviously significant in the end.


So, right. I mean, we don't have a specialized math portion of the brain, but I think we have general intelligence, HGI, I mean, evolution has given us various modules of thinking, right? So there's a vision processing module, and there's an amygdala, right? And there are certain problem-solving modules that I think, as primates and so forth, we had to develop, and how to run away from predators and whatever. But we also have, there's something in the human brain that lets us repurpose some of our modules for other tasks. So, I talked to different mathematicians, and some are very visual mathematicians. They have somehow repurposed the visual center to do mathematics. And some of that have repurposed their verbal center, and some have repurposed some of their fight or flight. So I don't know. It's somehow, we have enough raw material in our brain, like raw capability, and some way to learn and refine skills. That lets us do complicated things. I mean, not just math, but art and so many things that evolution doesn't directly give us the ability to do. But we just have a lot of raw material in our brain that doesn't directly give us the ability to do. But we just have a lot of raw primitives somehow, the cognitive primitives that can be harnessed in emergent ways. Somehow, we, our intuitions about logical correctness are extremely, extremely accurate. When the formal system, the formal representation of these arguments is unbelievably complicated. I'll say that this requires training, by the way. As I said, you talk to an undergraduate, their intuition of formal correctness is always insanely accurate. I think there's also a social intelligence component. I mean, we are good at math because our society has a collective institutional knowledge, and we teach our younger generation, and we have books and pedagogy and so forth. If you just sort of clone humans in a vat and set them on an alien planet, it's not clear that they would have anywhere near the type of math skills that took us 3,000 years to develop. It's not because our genetics is that much better, but there is a super intelligence beyond the individual human unit, and that's a large part of where the skills are coming from. Plus, even things like common language. Language is a technology, just like AI. The fact that we all speak a mutually intelligible language is hugely important. We cannot do math without it.


Awesome, thanks, Daniel. Okay, last but not least, another member of our technical staff, Shimon Sidor.


Hello. Yeah, so my question is, could you give an example of a math problem such that you assign around 5% probability that it will be solved by LLMs next year? Like the type of problem, it doesn't have to be a specific problem. There are sort of degenerate cases. If you want to modify two 1,000-digit numbers.


This is a problem that humans can't do, barely, okay, but even just plain old-fashioned calculators can do. So, I mean, you can cook up artificial problems that LLM could do.


I mean, I think we may have to start inventing new categories of problems to work on because they are particularly amenable to an AI-assisted paradigm, and we don't really know what types of problems these are yet. I mean, one parallel I can give is that about 10 years ago, I started getting interested in crowdsourced mathematics, so where you post a problem on the internet and you collect contributions from many different people at various levels of mathematics.


And initially, these projects are called polymath projects. So initially, there was sort of this dream that this would be a new paradigm and it could solve huge unsolved problems in mathematics. What we found was that there were certain specific types of problems that were amenable to a crowdsourced approach, but a very small fraction. Problems that you could break up into lots of modular pieces, where each piece did not require too much expertise, so different communities could work on different pieces, and ones for which there was some sort of overall good metrical progress, like sort of score function that kept going down. There's a couple of other features like that. So there was a certain narrow sort of criteria. And there were a few successes, you know. So there was one annals paper that was a polymath project, the very first one. It did actually come up with a new type of proof of an important theorem. But it didn't scale very well. One thing we also realized was that the human moderation needed to run these projects was quite intensive.


Maybe one day there'll be crowdsourced AI-assisted projects that maybe, you know, also AI-motivated to manage the chaos. But these are things that we could only work out after we tried quite a few of these projects, and then we drew conclusions. So I think we're only going to ask these questions empirically. I think we can speculate, but pretty much all speculations would be wrong.


Thank you, Simone. Okay, I think that was perfect timing. I want to say sincerely, Terry, thank you so much for joining us. You really worked hard tonight. You've been talking for an hour and a half, answering all of our questions. There were more than 100 OpenAI staff members that showed up tonight. And I really think that's testament to the way that you said you appreciate being a teacher of math. I think there are a lot of people here, whether informally or formally, have been motivated by your teaching. So thank you so much for spending your time here with us. And Mark, thank you for being an amazing facilitator of the conversation. Ilya, Daniel, Jakob, thank you so much, guys, for joining us.


And before we leave, I just want to tell everybody a little bit about what's coming up. We only have two more events in the OpenAI Forum to take us through to the end of 2023. Next week, we'll be switching gears a bit, and we're going to have Professor Dr. Ahmed Elgamal from Rutgers. He's going to take us on a journey of the history of AIR from the uncanny valley to prompting gains and losses. I hope you guys can join us for that.


And then the final event of 2023, we'll close out with another take on the future of work with Chief Economist from LinkedIn, Karen Kimbrough. And she'll present her findings reflected in the report, Preparing for the Workforce for Generative AI Insights and Implications. So all of us, we're going to be able to re-watch this on demand and share it with anybody in our community. If any of you have referrals that you'd like to add to the community so that they can watch this video on replay, and same to you, Terry, if there's ever anybody, any of your students that are interested in having a seat at the table with OpenAI and contributing to our conversations, I'm happy to invite any of your students or your peers. And I hope to see you guys all really soon. Again, Terry, thank you so much for joining us tonight. I hope everybody has a beautiful evening and I will see you all again soon, I hope. Okay. It's a pleasure. Good night, everybody. Good night.