Connecting InDesign to URL-based Assets
Silicon Connector is enjoying huge popularity, and as we build out more and more implementations (12 Connectors and counting!) the product is becoming more clearly defined, while the product roadmap is also taking shape. While the main feature of "connecting InDesign to URL-based assets" is itself quite enough of a product to save large authoring groups immense amounts of time, the "nice-to-have" features have taken on a life of their own, and become common to most new implementations. Here I will clarify the ways the product definition is being extended, now and into the future.
I hope to explain:
- What we originally meant by the term "Silicon Connector" and how this was consistently rather poorly explained by us, and in turn how it was often misinterpreted by the world.
- What we mean now by "Silicon Connector". How to understand what this product is, and what it does.
- Where the product and its many variants (AEM Connector, the InDesign Plugin for Flight, the Widen InDesign Plugin, WebDAM CC Connector, etc.) are headed.
In places, this post quotes heavily from the original blog post about Silicon Connector that came out when we first announced this product in 2010. At that point of time we had a narrow perspective on the product. Six years later, it has grown quite organically and evolves in response to feedback from thousands of users worldwide, as it connects InDesign to a diverse and growing array of over 10 Digital Asset Management systems.
The original meaning of "Silicon Connector"
"When I first met InDesign, in 2000, I (like many users then and since) thought maybe URL-based assets would be a feature that was already part of this then-new product: wouldn't it be cool to work with web-based assets? But no, it wasn't part of the product, and such a feature wasn't necessarily essential to getting work done."
Thus started my first explanation of Silicon Connector to the world six years ago, and right there is the original meaning of the name "Silicon Connector": the product that enables URL-based linking to assets from Adobe InDesign. I went on to explain how we at Silicon Publishing exposed this core Adobe-built technology to the world to do just that: enable direct, URL-based links from Adobe InDesign to files residing on Web servers or in Digital Asset Management systems (DAMs).
What Silicon Connector Was in 2010
In its original meaning, Silicon Connector was just our C++ plugins to provide this functionality. We were making URL-based links a reality so users can keep each asset in one place identified by a URL, and share or move InDesign files as desired without having to move the assets as well. While this is one very important and special piece of what we mean by "Silicon Connector", we are now finding it necessary to broaden the definition. This is due to the fact that Connector has been consistently misinterpreted by the world, as well as because the features of the product have evolved so much since then.
At its core, "Silicon Connector" is/was very much like what we offer currently for AEM Connector: low-level plugins that make URL-based links work from within InDesign.
At that point in its life (2010), Silicon Connector had two incarnations, built entirely in C++, providing exactly that functionality, limited in scope but deep in its impact. When you dragged and dropped an image out of the AEM DAM or MediaBeacon, our Connector (tailored for use with these two DAMs) would create a URL-based link.
Then you could share just the InDesign file with co-workers. With access to the same assets in the DAM, they could then load the InDesign document and those assets would simply be there, without any relinking or asset duplication.
What Silicon Connector Was Not
At that time, Silicon Connector did not have a UI of significance, although it did make minor changes to the Links panel in InDesign. For example, the AEM Connector would show which rendition had been selected, while the MediaBeacon Connector had an icon for toggling between low-res and high-res versions of an asset. Not much beyond that, and it was clearly an InDesign-only product as well.
The big thing that was missing was asset navigation from within InDesign, which is now a much more common product feature. Instead of providing navigation from within the authoring environment, the older version forced users into the interface of the DAM itself, toggling between InDesign and the browser where the DAM interface had been loaded, and dragging from the browser into InDesign to instantiate links.
Beyond that, Connector at that time only supported InDesign, as it had no relationship to the other Creative Cloud apps, such as Photoshop and Illustrator.
Since then, product scope has broadened in two important ways: asset navigation, and support for Creative Cloud apps. Both are features of many Connector-ready DAMs. Of course it is continuously improved and other natural enhancements were implemented: SSL, Single Sign-on, Proxy management, greater configuration/extensibility, etc.. But the asset navigation and Creative Cloud support stand out as really re-defining the product.
Beyond the features that made sense to us and were natural to the product, we were surprised to hear from customers who, upon hearing the word "Connector", managed to conjure up a slew of features they just knew must be part a product of that name. It turned out that there were more of these than I imagined possible, and I hear of new interpretations of "Connector" (a powerful term!) every day. Following are just a few perspectives.
What people thought "Silicon Connector" must be
Perhaps we picked a name that was too general. Because our audience knows that we are InDesign experts, when they hear the term "Connector", it's natural that they think of anything that you might want to "connect" to InDesign. This naturally includes:
- Relational Data
- XML, JSON
- Web Services
- Document content (such as Word)
- Graphic Assets (occasionally someone would get it right)
We heard clients saying "This is great, you did it! Now when I update the price in my InDesign catalog, my Oracle database is updated automatically. We definitely want your Connector!"
And this was rather painful, because we actually do connect InDesign to all these things, just not with the tool that we call Silicon Connector. Our far more expensive & capable Silicon Paginator product does the really complex things such as robust data-driven publishing from XML and relational data sources, but it is not the simple "plug it and run" "same for everybody" type of product that Connector is: therefore I would have to dramatically temper the expectations of clients who misinterpreted the initial news of the product.
It is magic, but magic within a very defined scope
Silicon Connector really worked with (and continues to work with) what InDesign considers "links" and these links do include some forms of data. To make things thoroughly confusing, there is also under the hood some two-way-linking functionality, i.e. if we worked hard enough we could extend Connector to update the system of record from a data-generated catalog. We might have done something like that, clients have certainly thought they wanted such a thing, but (a) it is generally a worst practice, and (b) it would be difficult to create and especially hard to maintain such an application.
Graphic assets, much more than data/content
The primary use case that DAMs brought to us was not catalog automation, or integration with robust content sources, or even integration with diverse external systems, but was instead focused on the assets that DAMs house, those things that InDesign can link to. Silicon Connector connects to almost everything that can be linked to from InDesign, doing so as a URL reference. It does just what InDesign linking does, and no more.
So for example, if you want to link to a text file, you can, but there are very limited behaviors regarding flowing that text through containers in some automated way, or conditional formatting based on its content. You can link to Snippets or ICML as well, but whatever sort of content you link to can best be considered a graphical asset more than a true data source: for example, we don't have "link handlers" that take the next step of intelligently flowing text or XML content through containers, as we do with Paginator.
We could extend Connector in that direction, but we really don't see Connector as a complete solution for database publishing (though it might do great with certain simple use cases). Connector, for the most part, merely connects InDesign to graphic assets. There are places that content-based or "data" linking can happen, but don't assume it will be robust. That form of connectivity might serve very specific use cases, but our Silicon Paginator product is what we choose for serious InDesign connectivity to data sources. We have an increasing number of Paginator/Connector workflows, and Designer/Connector works too. Having Silicon Connector makes any form of InDesign authoring more efficient, and while it doesn't automatically implement database publishing or web-to-print, it makes the InDesign dimensions of those things easier and more powerful.
What Silicon Connector is Today: Two New Functions
While we have not extended Connector to offer pagination features or robust connectivity to structured data/content sources, we have extended it substantially in two other directions. Based on feedback from users, we discovered two things that really enhanced this product:
1. In-app navigation of assets (an asset browser right within InDesign)
Many users wanted to see their assets from within InDesign, rather than having to toggle between the UI of their DAM and InDesign. We didn't jump on implementing this initially, as it seemed fairly involved to re-create all of the navigation, search, and other functionality that was already available in the DAM, but we were forced to do this when we first encountered a DAM that didn't offer an easy form of UI customization - namely Box.
We had experience building these kinds of in-app asset browsers; our Fotolia CS Extension had let users search, or even buy, Fotolia images without leaving InDesign, and we were adept with the new CC Extension technology. When Box approached us for a Connector, they didn't offer to tailor their generic UI to support it. Instead, we built an asset navigator right there, inside of InDesign. Other DAM partners and their clients clamored for the same thing. Of the four new Connectors currently in development, all of them offer navigation/search, so this seems to be a trend.
2. "Edit Original" workflow and DAM navigation from Illustrator and Photoshop
Beyond asset navigation, there was also a common request for what is known as "Edit Original" functionality, for InDesign users who reference Photoshop and Illustrator assets. With old-school vanilla InDesign, before the Connector modernized it, you could select an asset and choose "Edit Original" to invoke the authoring tool it was created in. With Connector in its early form, this would not really work. Instead, you would download a .PSD or .AI file from the DAM, edit it, re-upload, and the link would update. Not the worst scenario, but clunky.
So we enhanced several of our Connectors to acknowledge the "in DAM" nature of Illustrator and Photoshop assets. With the WebDAM CC Connector, for example, you can now edit such referenced assets and they will be correctly interpreted as assets residing in the DAM, not the local file system. Edits are checked into WebDAM, so there is no need to leave the Creative Cloud to do anything.
A wonderful side-effect of building this "Edit Original" functionality was that we also had to build some form of UI into Photoshop and Illustrator to facilitate it. In the end, we added functionality to navigate the DAM from within those applications. Important to note: while both Photoshop and Illustrator have some concept of linking, neither has the deep URL-based data model that leveraged with InDesign for our base "Connector" plugin. The level of DAM connectivity (for Creative Cloud products other than InDesign) is only suitable for single, self-contained assets. Because InDesign is the "hub" in most authoring workflows that we encounter, this has usually proven to be acceptable.
"A Connector from the Adobe Creative Cloud Apps to URL-based Graphic Assets"
So the product has finally taken shape. A good baseline statement encompassing what is now would be: "A Connector from the Adobe Creative Cloud Apps to URL-based Graphic Assets."
Connector now offers some or all of the following (depending on the specific DAM implementation):
- InDesign support for true URL-based links.
- Asset navigation, download, and upload from within InDesign, Photoshop, and/or Illustrator.
- "Edit Original" workflow where an InDesign document is "DAM-aware" when working with linked .PSD and .AI files.
But the question remains: what is Connector "not"? Well, that may change in the future, and that doesn't mean we can't "connect" in many other ways, bringing many other things to InDesign. I have bought pizza through an interface in InDesign, so it seems as though there may be no end to what we could connect to, yet we find that even a subset of the above functionality has great ROI for authoring workgroups.
Twelve DAMs and Counting
I was very excited about this when we first got it to work, but soon after very let down by the initial cool response from the world. I almost gave up, actually, because what I knew could be a game changer for creative workflows was so easily dismissed and overlooked by some very prominent leaders in the DAM space. The dream was kept alive by the first users of the software, who loved it, and encouraged us to keep at it. Eventually, some DAM luminaries such as Jason Bright showed enough enthusiasm (and sold enough of the product) that it was validated. When we encountered the fourth or fifth DAM with interest, we realized that yes, it's just as meaningful as we thought. Now that we have thousands of happy users around the world, the value is getting obvious, but it took tenacious effort and faith, and the support of a number of great partners, to see it through.
At the moment there are twelve connectors, and we are in discussion with almost every major DAM vendor. There is no similar form of InDesign linking functionality that comes remotely close in terms of performance, so we expect every significant DAM (certainly any interfacing to Adobe technology) to eventually have a flavor of our Connector available. I am thankful that we attained the initial breakthrough through the amazing work of Michael Easter, and that we have extended the definition of the product through the work of the master of Adobe extensibility, Olav Martin Kvern.
Connector will continue to grow, and although its definition may expand again in the future, as it stands today there is no better technology for connecting InDesign and other members of the Adobe Creative Cloud to the world's leading DAMs.