How Can I Effectively Produce Data-Intensive Business Documents?

Data-personalized print documents and their PDF cousins are essential business tools. How do companies produce these economically—and at scale?

We’ve had “business documents” ever since Sumerian accountants started using clay tablets as a sales and inventory management system. (In those days, a system crash was the sound of clay shattering on the floor.) Over the millennia, humans have struggled to find the best ways to record important information—and be able to find it later and communicate it to others. In a previous blog, we looked at this challenge from a marketing perspective, but the same issues plagues the creation of customized business communication.

The problem is twofold. First of all, contrary to all the “paperless office” hype, the reality is that fixed-format documents (whether printed on paper or in PDF), are still the most effective way to convey business data meaningfully. Studies show that paper-based reading yields better comprehension outcomes than digital-based reading, especially for explanatory, non-narrative content (this blog notwithstanding!)

The second part of the problem is the ongoing explosion of big data in business. Yes, print/PDF is the ideal output medium,  but there is just too much data to customize that type of output cost-effectively. What businesses need is an “engine” that can…

• Insert the right data into the right place, at the right time, in a well-designed business document, without breaking the design
• Do so accurately and securely, for hundreds of thousands, if not millions of recipients, and
• Do so with little to no costly manual effort.

What are the technical challenges of producing personalized business documents at scale?

The concept of variable data printing (VDP) is not new. Like the mail merge function in most word processors, VDP can insert the contents of a data field into a template placeholder, producing as many personalized document versions as there are records in the database. In theory, this sounds simple enough, but in practice there are major technical challenges:

• Often, the open-source page rendering engines used by some VDP systems can produce poor or inconsistent visual results—or even distort how the data is displayed.
• Very large, complex data sets can bog down some VDP systems.
• Complex, multi-page templates (e.g., those with multiple fonts, tables, layers, and high-res images) can overwhelm many traditional VDP systems.

All of these issues can be solved with a professional-level page rendering engine. For complex, typography-intensive print layouts, this means using Adobe InDesign Server, a core component of Silicon Publishing’s Paginator application.

Handling privacy and data security issues

Creating data-customized business documents involves serious responsibilities. For healthcare entities in the U.S., strict HIPAA rules govern patient privacy. For financial services companies, law firms, and government agencies, there are major privacy and data security concerns—with dire potential consequences if they are mishandled.

In the past, some VDP systems have attempted to add data security features to address this concern. However, for many, this has proven to be unrealistic, especially if the publishing system is SaaS-based. Instead, where security is a high priority, the solution is to insist that the publishing system be hosted locally, taking full advantage of the organization’s existing data security infrastructure. Paginator is often deployed in extremely security-sensitive environments, including the intelligence community.)

How important is typographical quality?

There are many document personalization approaches for highly structured documents, including technical or training materials. For these, XML and the open DITA standard are adequate. But if the documents are typographically demanding, or require precise control of images and other page elements, then a more robust rendering architecture is required.

There are good reasons for this:

• Pleasing, consistent typography is an affirmation of the organization’s brand integrity. Even when the connection is a subconscious one, the aesthetic qualities of well-composed type reflect well on the sender. For example, good typography can reinforce the organization’s focus on consistency and quality.
• Typographic control is vital in guaranteeing document accessibility. For example, the validation process in Paginator ensures that all body copy is rendered in a pre-determined point size—one of many accessibility requirements.
• Typographic flexibility is also crucial in complex documents. If the data includes text strings of different lengths, or a requirement exists to generate versions of a document using non-Roman fonts, then the rendering engine must be able to move and/or scale elements to maintain a professional appearance. And it must do so automatically and at scale.
• Type and other elements must be data-aware. Good design includes an awareness of context—how different elements are related to each other, like photos and their captions. Paginator uses InDesign “snippets” to maintain and apply these contextual cues for each personalized document.

What is the business case for high quality, data-personalized business documents?

We have come a very long way from the 1980s, when WordStar and WordPerfect first launched mail merge features in their now-defunct PC programs. Today, vast and complex data sources can be used to create millions of personalized, highly informative, and aesthetically compelling documents. If executed well, these pieces foster greater brand loyalty and business retention—not to mention conveying valuable information about… just about everything else.

A previous blog covered the marketing aspects of data-personalized output, but the line between customized marketing and “business” documents is becoming more blurred. Routine, customized statements and reports increasingly include “trans-promotional” material, using the same data management and output rendering approach.

No matter the format, using data to produce printed or PDF output that is visually compelling, accurate, and practical is a win-win for any business.

Frequently Asked Questions

What are some of the challenges in producing high-quality, personalized business documents on a large scale?

Many rendering engines used by VDP systems can produce poor or inconsistent visual results. Also, typographically complex, multi-page templates can overwhelm traditional VDP systems.

How can personalized business documents be produced securely?

Rather than the vendor-managed SaaS option, you might explore hosting within your own environment. Bringing this in-house onto your own servers gives you greater control over data governance, and may better align with your security and compliance requirements.

How important are typography and aesthetics in business documents?

Good typography and pleasing aesthetics are very important in any customer-facing business document. These qualities reflect the organization’s brand identity, while ensuring accessibility and reinforcing the perception of consistency and quality.