Converting Content and Coding DITA with Claude AI

For several years a key focus of artificial intelligence (AI) as applied to technical documentation has been on using it as a basis for chatbots. I worked for a consulting firm that touted the benefits of structured content (DITA) that could be applied this way. But somehow, I think the more obvious approach of using large language models (LLMs) to produce structured content directly got missed. And while this idea seems to have finally caught on—see Oxygen’s AI Positron Assistant or Intuillon’s DITA.ai—I haven’t seen anyone share their experiences with AI when it comes to working with DITA. So, I decided to put an LLM through the paces of helping me convert existing content to DITA.

After testing several online LLMs, I settled upon Claude.AI, as produced the best results. It is from Anthropic, and I put it through its paces using its free plan. For my sources I took a couple of old manuals from Project Gutenberg: one for the original Model T Ford car, and a WWII-era manual for a U.S. army-issued version of the Harley Davidson motorcycle. I took each topic Claude generated and pasted it into Oxygen in Text (code) mode to validate it.

My approach was simple: I would copy a topic’s worth of content from the original manual and ask Claude to convert it to a specific DITA topic type, while also asking it to provide a short description.

Here are my findings.

Pro: It is Fast

After clicking the “Send” button, Claude IA immediately started generating DITA-based code based on whatever I had asked it to tackle. Even when the content submitted was dozens of paragraphs long, Claude immediately started spooling DITA code outlines. The longest topic took no more than 20-30 seconds to appear. Even though it was not always perfect, it did its job quickly, much faster than any human could.

An Example of Claude Creating DITA Code On-the-Fly

Claude was also good at taking an oversized “supertopic” (such as an extended troubleshooting section containing multiple causes and remedies) source and breaking it down into smaller, more concise topics.

Claude Breaking Down One “Supertopic” into Several Troubleshooting Topics

It’s also worth pointing out that because it is fast, it can churn out a lot of topics quickly. Even in cases where the results need tweaking and then made more consistent with each other in terms of coding style, quantity can be considered a type of quality.

Con: Lots of Back-and-Forth

While there were a lot of occasions where Claude got things right on the first try, there were many times when the DITA code created did not validate. My approach when validation errors happened was to first ask it to re-validate its code to see if it could figure out where it went wrong. Often it was able to figure it out, but there were several times where it would come up with substitute code that was incorrect in a different way. Troubleshooting topics were problematic for Claude, often taking several runs before getting it right, and there were several instances where it just seemed easier for me to correct the code myself.

Claude Validates and Still Gets Order Wrong

There were also a few cases where it invented new DITA tags, and it never seemed to get the hang of any element that used camel case, like <troubleSolution>, always making it all lower-case.

Claude Using a Non-Valid DITA Tag

Pro: It Excels at Re-writing

There’s no getting around it: the WWII-era Harley Davidson manual is a poorly written document. While it is thorough—it lists 86 things to check for preventative maintenance—it meanders, it is pedantic, and is the antithesis of concise. I took a stab at converting this document myself several years ago and gave up because it is such a bugbear. Claude does not complain when given large chunks of this content to digest, and its rewrites are genuinely concise. It does not reproduce the content word-for-word (this could be a problem in some circumstances), but instead intelligently restructures and rewrites the content that is more easily digestible.

The same goes for restructuring content. After reviewing the code I would sometimes see ways that things could be improved from a DITA perspective. All it would take was to point it out to Claude, and the AI would make the requested improvement on the spot.

Claude Restructuring by Request

Also worth noting is that Claude came up with good short descriptions for each topic it created. I couldn’t find much to fault in the text that it came up with.

Con: No Holistic View

Any LLM is inherently stateless, meaning that it doesn’t “know” what is happening from one session to the next. This would seem to go against the “back-and-forth” I just talked about, and while the AI could comment and correct code it worked on in each session, it had no inherent understanding or connection between the individual topics it created. This led to inconsistencies in the ways that it approached its conversion process. One task topic might use the “strict” model, and the next would use the “loose” model (and more than once it tried to do both in the same topic). This also meant that there was no overall understanding of the document, though this is mitigated by the DITA standard being modular in nature. Oddly enough this is another hidden strength of the DITA standard, as its structure makes it easier for an AI like Claude to put together topics that make sense in isolation.  

And while an AI like Claude may not have an inherent understanding of the topics it creates, you can ask it to structure a basic bookmap for you, so long as you describe the structure and give it the order of the filenames.

Claude Structuring Topics Into a Basic Bookmap

Not only that, but you can also ask Claude to replace the topicrefs with conkeyrefs and have the keys placed into a separate file referenced by the bookmap:

Claude Restructuring the Original Bookmap Using Keys

I know this section started off as a “con”, but in the hands of an experienced technical writer an AI like Claude can be guided in creating DITA content that can greatly help the automation of converting non-structured content to DITA.

Conclusion: AI is Usable Now as a Tool for DITA Conversion Efforts

While there are several drawbacks to using AI to convert existing content, the speed at which an AI like Claude can produce DITA topics on demand is hard to beat. It still needs a human in the loop to check and then tweak the results, but the process is fast and accurate enough to beat all but fully customized conversion programs for outputting DITA.

The fact that the AI reinterprets and rewrites the content may be a problem for some, though you can just ask the AI not to. In most cases, it did a better job than the original, and I would argue that the more messed-up the original wording in the source document, the better an AI like Claude rewrites things. And you need to validate the structure of what it comes up with, as the AI is prone to inventing new tags and inserting valid DITA tags in invalid places.    

Given some additional user interface tweaks and trained more rigorously on DITA code, an AI-based conversion process would be a terrific tool for any technical writer. As it is, an AI like Claude is already most of the way there but needs a person to check and guide its efforts.

I am going to continue my efforts to convert more content using AI, so watch this space for updates when I post the results to GitHub. And if I discover anything new with Claude or any other AI I use for the conversion efforts, there may be a “Part 2” to this article.