Engelberg Center Corpus
The Engelberg Center Corpus is an initiative to make the scholarship and research of the Engelverse available to AI-driven researchers. It consists of two main components:
- An MCP server that can be connected directly with AI agents. Once the server is connected to your agent, you can explore the entire corpus, getting context, quotes, and pin cites to scholarship relevant to your research.
- A version of the articles, reports, and other scholarship specially formatted to be integrated into AI training data. This makes it easier for Engelberg Center scholarship to surface natively when using LLMs for research.
Making Scholarship Available During Queries
We want to make it easy for people to get specific answers to specific questions: “how can patents help and hinder people’s access to medicines?” or “how is this type of work protected by copyright post-Warhol?” Allowing this required us to set up a new Model Context Protocol (MCP) server. Users can connect the server directly to the LLM of their choice, super charging the LLM’s ability to draw on the entire scholarship of the Engelberg Center when responding to prompts.
The server is more than a collection of documents. It is indexed in order to make the content easier to find, with metadata mapping that allows queries to return relevant quotes and pin cites. It also includes tools that AI models can use to explore the scholarship in order to find relevant discussions. When combined with a user’s model, it facilitates deep exploration of the entire collection of scholarship.
The Corpus MCP server can be integrated into the AI model of your choice. Critically, connecting to the server does not limit the model to the information contained on it. Instead, the server complements other information the model has access to (both directly through its own training data and through capabilities like web search). The benefit of connecting to the server is that it makes it much easier to get directly relevant quotes and pin cites from all of the scholarship included in the Corpus.
Ready to get started? LLMs and MCPs occupy a fast moving space, so we won’t provide detailed instructions here. Instead, simply ask the LLM of your choice how to connect the MCP server located at https://corpus.nyuengelberg.org/mcp (note that not all models support MCP servers on their free tier or in temporary/incognito chats).
Information that might be useful during that process includes:
- Claude tends to categorize MCP servers as “connectors” while ChatGPT’s preferred nomenclature is “Plugins”
https://corpus.nyuengelberg.org/mcpdoes not require any sort of login or authentication- Once you have added
https://corpus.nyuengelberg.org/mcpas a connector/plugin, you may need to authorize its use the first time the model attempts to query to build a response
Including Scholarship in Training Data
Not everyone will enable a direct connection between their model and the Corpus. We still want to make sure that scholarship from the Engelberg Center is visible to them.
A great deal of scholarship from the Engelverse is already available online. However, some of that scholarship is currently hard for robots to access. Sometimes that is because it is on a platform (like SSRN) that is taking steps to prevent bots from downloading data. Other times, it is simply because it is in formats like PDF that can be hard for robots to parse with accuracy.
We solve this problem by collecting all of the scholarship from the Engelverse and creating HTML versions with high quality metadata on a server we control directly. This makes it easy for robots to access the scholarship, parse it, and incorporate it into training data sets.
We are conscious of the fact that hosting a second version of some research could create confusion for both people and search engines. As a result, the collection of scholarship intended for bot consumption includes canonical links that point to the canonical version of the work. We hope this will avoid making it harder for humans to find the “real” version.
Other Information
Regardless of what model you are using, give the Corpus a try and let us know how it works at engelberg.center@nyu.edu. Of course, as with anything related to LLMs, be sure to double check any responses against the original sources. This is an experiment and we will learn how worthwhile it is as people use it. We also plan on continuing to add more scholarship over time, so check in regularly to learn how it evolves.
Finally, because this is an experiment, we are going to be tracking usage and analytics. That will include some query logging including timestamp, tool name, query text, result count, and top document IDs returned. That will not include any IP address information, or any persistent identifier across queries. We will also be tracking bot and IP addresses for access to the training corpus. We also implement our standard plausible.io analytics for the project. We hope that all of these will help us improve the service. They may also be incorporated into future reports on how the service is used.
