GPT-3 appeared last year:

https://www.forbes.com/sites/bernardmarr/2020/10/05/what-is-gpt-3-and-why-is-it-revolutionizing-artificial-intelligence/

Given that was not made openly available like GPT-2, others made an open source version, hence we have GPT-J and GPT-Neo, the latter being relevant for here.

I have taken the 1.3B parameter model and thanks to HuggingFace and their model implementations, used this Colab notebook.: https://colab.research.google.com/drive/1H2mUZsYhel4g5ZUOmZHtYIaP4gSW5_ow?usp=sharing#scrollTo=eBIcgwE1kVQK The 2.7B parameter doesn’t seem to work here – runs out of memory. Not surprising as it is 10GB.

I thought a fun holiday project would be to use this to generate soom module type descriptions.

There are certain limitations of how long the text prompt for the model to predict from can be, so for a first try I took the three entryway possibilities for the Tomb of Horrors.

Here’s the output:

4. TOMBS OF HORRORS: The rooms of the torture chamber can be
   visited only if players are in the same chamber, using
   special note if they are in the same room.
   At the end of the hall, 2 pits lead to:

5. WELCOME TO THE TOMB OF HORRORS: The rooms of the wizard’s work room
   can be visited only if players are in the same room, using
   special note if they are in the same room.
   In the center of the room (the only room not full of
   druids and other such creatures), you will find the statue
   of a great warlock, his robe stained with the blood of
   victims. This statue is the work of the warlock’s master.
   The wizard’s work room is otherwise empty.
   Torture chamber area. At the end of the hall, 2 pits lead to a

6. THE TORTURE CHAMBER: If players are in the same room
   (or in the same space), the torturing chamber can be explored
   independently.
   One end of the chamber is a tiny chamber whose floor is
   painted in a pattern of little figures of evil or good
   design, and whose walls and ceiling are also painted in the
   same pattern. The figures seem to be moving a weight
   over some sort of anvil. The floor may be of lava or covered
   with mud. Either of these may hold the weight of a figure with
   a hand upon it. (See special note regarding the pits at the end
   of this paragraph.) The wall on the other side of the chamber
   opens up, and a chamber in the wall appears with a pit, a
   stone wall, and a lever. A prisoner is taken into the pit. The
   prisoner may be tied or, if it is tied, the prisoner is
   released if the lever is pulled, and pushed into the pit again.
   An assistant, an orc or a human being, lies dead by the lever.
   This must be done by all players in the same room. The
   chamber (or it may be filled with water) is now filled
   with water, and a person is drawn into it and the lever is pulled. The
   prisoner is still alive, as the lever is slowly pushed upwards,
   and it is possible to see that the prisoner’s body is
   thrown back down into the pit again and again as
   it is released from the lever’s grip. If you were to use
   a light, you would notice the number of men (or women)
   released into the water from the torture pit, and their
   number would be shown in the water as an illustration to
   the left. If you were to use a sword, you would notice that
   the number of men (or women) released from the

As you can see, that is pretty good for autogenerated based on a few paragraphs prompts from a generic model trained on close to a terabyte of stuff from the internet, and not tuned to the experience as in something like AI Dungeo – https://play.aidungeon.io/. It even recognised the room numbering of 1, 2, 3 and continued on. It is generating text, so not going to finish sensibly all the time. Not much work for a DM to join those together and edit a sentence or two though.