Hey!
I am presently working on an MCP toolkit that connects procedural generation applications together to speed up development on Unreal Engine. I released an MVP for a hackathon last weekend which included connections between Gaea, Unreal Engine and PCGExtendedToolkit. (The repository still needs cleanup and review.)
Right now, I am focusing on improving the Gaea interaction by adding more tools and elaborating a pipeline and architecture to get better results from prompted AI. This includes building a RAG among other things, which implies going through all the documentation and formatting it in a digestible way for AIs.
I am also in the process of exploring whether adding some kind of DSPy (GitHub - stanfordnlp/dspy: DSPy: The framework for programming—not prompting—language models · GitHub) might be worthwhile.
If anyone is interested to contribute (the project is open source, here: GitHub - zajalist/hayba: AI-powered terrain and procedural generation — HaybaGaea + HaybaPCGEx · GitHub) or has suggestions for how to improve the design and architecture of the tool, do let me know!
That sounds like a great direction for UE users.
One thing worth keeping in mind with Gaea 2 is that automation and pipeline integration seem fine, but AI-related use is where to take a lot of care, the EULA is fairly helpful to navigate this, and would make a good reference to help refine end goals.
For the docs side, I’d strongly suggest using the official LLMS.txt / Markdown pages from docs.gaea.app
Much cleaner and EULA positive than something like scraping the site or building from software/assets directly.
I am currently updating the docs weekly publishing is typically after a review period. Thatgives you a cleaner and safer source for an AI assistant or lightweight retrieval system.
Right now it has mostly just been a playground of me trying different things and seeing how it works out, especially given hackathon constraints. The architecture is just LLM → RAG (inference) → OUTPUT.
Thanks for the llms.txt suggestion. I did not read fully the EULA for the scope of the hackathon so apologize if I did something in a grey area. I do however have a question:
Sometimes the logic of the node graph makes sense but the parameters and some combinations lead to really bad visual results. The more complex a graph is, the more it becomes prone to bad results.
I was thinking of moving away from that architecture so that the model itself can understand the visual result of what it is producing and whether it fits the intent/use-case of the user prompt.
Something like: DSPy (LLM) → Convolution Neural Network (heightmaps, slope masks, etc…) → Graph Neural Network (Inference) → Cross-attention, etc… (Inference) → OUTPUT
But I would need a lot of data for training and I am not a legal expert, but the EULA seems quite explicit and restrictive in the use-cases of AI so I wanted to confirm with the QuadSpinner team that this is something fine to do and, if not, if you guys could share suggestions of alternatives.
The goal of this project is to have a library of toolsets where machine learning models among other things can facilitate to scaffold through a variety of procedural tool within the Unreal Engine production pipeline.
I would send in a written request for your use cases (Define Scopes), and see if you have any wiggle room to work with and or can come to some other agreement.
The EULA states
CNN/GNN/cross-attention isn’t allowed without permission.
Automation seems more reasonable but it’s mostly just for internal use.
And then the only other parts I can see are licensing requires Pro or Enterprise.
See if that helps you out some.
1 Like