Category: Integration

Connecting a DAM, a PIM, a storefront and everything downstream without building a nightly sync job that nobody can debug. API design, headless commerce, reference-over-copy patterns, and the integration shapes that survive a replatform. Written for the people who have to make two systems agree.

  • DAM for developers: the API-first path

    DAM for developers: the API-first path

    The evaluation document has forty rows and thirty-six of them are about the interface. You will use the interface twice. You will use the API every day for the next four years.

    Short answer: an asset platform is worth adopting when three things are true. You can upload and get back a stable identifier in one call, you can construct any rendition as a URL without a round trip, and you can search and manage assets over a documented API with real SDKs. Everything else is convenience. Those three determine whether it disappears into your stack or becomes a system you maintain.

    The three primitives

    Upload returns an identifier. One call, a file or a remote URL in, a stable identifier out. The identifier must be yours to choose or at least yours to predict, because your product records will store it and they will be written before the upload completes in at least one code path. Upload presets are the mechanism for making an upload carry a policy rather than a pile of parameters, which matters as soon as more than one thing uploads.

    Renditions are URLs. Given an identifier and a set of parameters, the delivery URL is constructible client-side, deterministically, with no API call. This is the property that makes the whole thing composable, and it is what separates a delivery platform from a file host with a resize endpoint. The transformation reference is worth reading not to memorise but to check that the parameter space covers what your design system needs.

    Assets are queryable. Not just listable. A search API that can filter on your own structured fields is the difference between an asset library and a bucket with a nice front end, and it is the thing you will need the first time someone asks for every image using an expiring licence.

    If a platform gives you those three cleanly, most integration questions have short answers. If it gives you two, you will be writing a service to paper over the third.

    The first call test

    Before any evaluation meeting, try to get from nothing to a working delivery URL. Time it, and note how many browser tabs you needed.

    The shape you want looks roughly like this: install the SDK, set one credential, upload a file, receive an identifier, print a URL with a size and a format parameter, load it in a browser. Cloudinary publishes SDKs across about seventeen languages plus a CLI, and the whole loop is a handful of lines in any of them.

    That test tells you more than a feature matrix, for a reason worth stating explicitly. The path from zero to a working call is the path every developer on your team will take, repeatedly, for years, and the path an AI coding assistant will take when it writes the integration for you. A platform whose first call requires provisioning, a support conversation and a bucket policy is a platform that will be integrated once, badly, by whoever drew the short straw.

    Cloudinary has done unusual work on that second audience specifically. It ships MCP servers and an llms.txt alongside the normal documentation, so an assistant retrieving context gets the real parameter syntax rather than a plausible guess. Whether or not you care about that today, it is a reasonable proxy for how seriously a vendor takes the machine-readable surface of its own product.

    Why day one beats month eighteen

    This is the part that gets argued about, so here is the argument without adjectives.

    The usual advice is that a media platform is something you adopt once you have a scale problem. That advice assumes the value is scale-contingent, and it is not. What you get from the first upload is structure: the asset has an identifier, a description, a version, and a defined way to render itself. What you get without it is a file in a folder whose meaning lives in a naming convention and in the head of whoever wrote it.

    The cost of not having that structure shows up immediately, not at scale:

    • The first crop someone needs becomes a file. By month three there are two hundred files nobody can trace to a master.
    • Image paths get hard-coded into the front end, because there is nothing to construct them from.
    • Nothing records what an image depicts, so alt text is either absent or invented per placement.
    • Nobody can tell which of two similar files is current, at forty assets just as much as at forty thousand.

    Then the migration arrives. Moving to a structured media layer at month eighteen means finding every hard-coded path across every codebase, matching every stored derivative to a master that may no longer exist, and backfilling metadata that was never captured. Doing it on day one costs an account and a URL helper.

    For a solo developer or a small team building an MVP, this is the whole argument. You are not adopting infrastructure for the scale you hope to reach. You are avoiding a migration you would otherwise schedule for the exact quarter when you can least afford it. The free Programmable Media tier makes the day-one version cost nothing, which removes the last reason to defer.

    A teal-edged core with six radial connector plugs, five seated and one amber plug held at an angle with the wrong key profile

    What to check that the demo will not show you

    Signed and time-limited URLs. If any asset is not public, you need signed delivery and you need to understand how signature verification interacts with caching. Ask early, because retrofitting it is unpleasant.

    Access control granularity. Be precise here, because vendor material tends not to be. Permissions in most asset platforms, Cloudinary included, are organised at the folder level rather than per asset. That is usually fine, and it is only fine if you design your structure around it rather than assuming per-asset rules you can query at runtime.

    Relationships between assets. Similarly, do not design an integration that depends on traversing arbitrary asset-to-asset relationships as a first-class API concept. Model relationships on the product side, where they belong anyway, as described in DAM vs PIM.

    Eventing. Notifications on upload, transformation and moderation, with retries and signed payloads, so your systems react rather than poll.

    Exit. Can you enumerate every asset and download every master through the API. The answer sets your migration cost, in both directions, and migration tooling that works on the way in usually works on the way out.

    Four teal cards in a short vertical sequence beside a much longer column of fourteen small amber cards descending past the frame edge

    When is a bucket and a CDN enough?

    When all of the following hold, and they hold more often than this page implies.

    • Assets are public, few, and rarely change.
    • One front end, one set of breakpoints, no marketplace or partner consuming your images.
    • No third-party rights and no expiry dates to track.
    • Nobody needs to search assets by anything other than filename.

    That is a real configuration and object storage behind a CDN serves it well. The threshold is crossed by the second consumer of the same images, or by the first licence that expires, and both tend to arrive without warning.

    What is worth avoiding either way is the middle option: a bucket, plus a resizing service, plus a hand-rolled metadata table. That combination has the operational burden of a platform and the capability of a folder, and the reasoning behind that judgement is in build versus buy for the product catalogue.

    What does the integration look like once it is right?

    Small. The product record holds an identifier. A shared helper turns an identifier plus a named shape into a URL. The front ends call the helper. A webhook updates a three-field read model when an asset changes state. There is no sync job and no staging directory.

    That is the whole thing, and the surrounding design is in wiring a DAM to a PIM for the catalogue side and media in a headless commerce stack for the front-end side. The asset-level capabilities it depends on are in what a DAM actually does, and the parallel set of questions for the product platform is in what to demand from a PIM API before you buy. If you are weighing this against assembling something from tools you already have, alternatives to buying a DAM and a PIM covers those honestly.

  • The PIM API: what to demand before you buy

    The PIM API: what to demand before you buy

    The API section of the demo lasts four minutes. Someone fetches a product, the JSON looks reasonable, everyone nods. Six months later you are writing a script that pages through eleven thousand products every night because there is no way to ask what changed.

    The four minutes were not enough. Here is what to ask instead.

    Short answer: eight questions decide whether an integration will be pleasant or permanent employment. Can it express your variant structure, can it tell you what changed since a point in time, can it page deterministically, can it write in bulk, is it idempotent, does it push events, does it expose the completeness state, and can you get everything out. Ask them before the contract, in writing.

    One: can the API express what the interface shows?

    This sounds absurd and it is common. A platform whose interface presents a family, a product and its variants may expose an API that returns a flat list of sellable items with the parent relationship implied by a shared prefix.

    Test it with your own worst case, not with their sample data. Take the most awkward product you have, the one with three variant axes and an incomplete grid, and ask them to model it and show you the API response. If the response cannot round-trip that structure, everything you build on top will be flattening and re-inflating it, which is where data goes to get lost. The structure this has to survive is described in attributes, variants and the SKU problem.

    Two: how do you ask what changed?

    The single most consequential question, and the most often answered badly.

    You want a change feed: give it a timestamp or a cursor, get back everything modified since, including deletions. What you often get instead is a filter on a modified_at field, which is nearly the same thing and fails in two ways. It misses deletions entirely, because a deleted record has no row to carry a timestamp. And it misses records whose modification was caused by a related record changing, unless the platform propagates timestamps, which most do not.

    Ask specifically: does a change to a variant update the parent’s timestamp? Does a change to a shared attribute update every product using it? If the answer is no to either, your incremental sync will silently miss updates and you will discover it through a customer.

    Three: is pagination deterministic?

    Offset pagination over a changing collection loses and duplicates records. It is not a subtle failure; it is arithmetic. If a record is inserted before your current offset between page four and page five, one record is skipped, permanently, with no error.

    You want cursor or keyset pagination, where the cursor encodes a position in a stable ordering. Ask what happens when a record is inserted mid-traversal. A good answer describes the guarantee. A vague answer means offsets.

    Also ask about the page size ceiling and the rate limit together, because they multiply. A hundred records per page against sixty requests a minute means six thousand records a minute, which is fine at ten thousand products and not fine at two hundred thousand.

    A twenty-segment strip with the first three filled teal and joined by cursor arrows, an amber broken bracket spanning the rest

    Four: can you write in bulk, and what happens halfway through?

    Loading a catalogue one record at a time is slow enough to matter during migration and during any bulk correction.

    Two properties to establish. Whether there is a bulk endpoint that accepts many records in one call, and what its failure semantics are. A bulk call that fails atomically is easy to reason about. One that applies the first four hundred of five hundred records and returns a partial error is workable if it tells you precisely which ones. One that returns a generic failure after partial application is the worst case, and it is common.

    Ask for the shape of the error response before you believe the answer.

    Five: are writes idempotent?

    You will retry. Networks fail, jobs get rerun, someone double-clicks.

    An idempotent write means sending the same request twice leaves the same state as sending it once. The usual mechanism is an upsert keyed on your identifier, or an idempotency key you supply. Without one, a retry creates a duplicate, and duplicates in a product catalogue are unusually expensive because they propagate to channels before anyone notices.

    This is also the reason to key writes on your own stable identifier rather than on the platform’s internal one. Your identifier exists before the record does, which is what makes upsert possible.

    Six: does it push, or must you poll?

    Polling a catalogue for changes is a cost you pay forever, and its latency is bounded by how often you are willing to pay it.

    A platform that emits webhooks on create, update, delete and state change lets you react in seconds and stop scheduling anything. Ask about the delivery guarantees rather than just the feature: is delivery at least once, is there a retry policy, is the payload signed, and does the event carry enough to act on or only an identifier you must then fetch.

    At-least-once delivery is normal and fine, provided your handler is idempotent, which is the same property as question five viewed from the other side.

    A teal emitter firing three darts at receiver plates, two landing cleanly and one amber dart fallen short of a tipped-over receiver

    Seven: is the completeness state readable?

    The completeness model is the reason to buy a product platform at all, so it is worth checking that the number is available outside the interface.

    You want to query “which products are not ready for channel X, and which fields are missing”. If that exists only as a dashboard, you cannot build it into your own operational tooling, cannot alert on it, and cannot include it in a launch checklist that runs automatically. The operational value of that query is set out in running product content as an operation.

    Eight: can you get everything out?

    Ask it plainly. Can we export every record, every relationship, every version and every asset association, in a documented format, through the API, without a support ticket.

    The answer sets your exit cost, and it is the question that most reliably distinguishes vendors who expect you to stay because you want to from those who expect you to stay because you cannot leave. A confident answer is a good sign about everything else.

    What about the general shape of the API?

    Less important than the eight above, but worth noting.

    REST with sane resources is fine and is what most of this category ships. GraphQL is pleasant for read-heavy front ends because it avoids over-fetching, and Shopify’s admin API is a well-documented example of the style if you want to see one. Neither choice determines whether the integration will work; the change feed and the pagination do.

    Authentication should be OAuth 2.0 or scoped API keys with the ability to issue and revoke per integration. A single shared key with full access is a finding waiting to be written up.

    Custom fields deserve one specific question: are they first-class in the API, with the same query and filter capability as built-in fields, or are they a bag attached to the record? Shopify’s metafields are a reasonable reference point for what first-class looks like. If your extensions are second-class, everything specific to your business will be the slow path.

    How should you actually run this evaluation?

    Not as a questionnaire. Ask for a sandbox account and spend a day writing the smallest real integration: load fifty of your own products including the awkward one, change three of them, and pull the changes incrementally. That day answers six of the eight questions honestly, and it answers them about your data rather than theirs.

    The answers then feed directly into the integration design in wiring a DAM to a PIM and into the field ownership list in single source of truth applied to product data. The equivalent set of questions for the asset side is in DAM for developers.

  • Media in a headless commerce stack

    Media in a headless commerce stack

    The headless migration goes well. The catalogue is an API, the storefront is a separate application, a second front end appears for the app, and a third for the in-store screens. Then someone notices the app is downloading two megabyte hero images to a five inch screen, and the fix is a ticket in three repositories.

    Short answer: in a headless stack the product API should return an asset identifier and a canonical delivery URL, never a set of pre-baked sizes. Each front end appends the parameters it needs at request time. That keeps the shape decision in the front end that knows the viewport, and the asset decision in the system that owns the asset, which is the only split that stays stable as you add channels.

    What headless changes about media

    Headless architecture removes the assumption that there is one front end. That assumption was carrying more weight than anyone realised, because it was also the assumption that let you decide image sizes once, centrally, at build time.

    With one front end, a fixed set of renditions is workable. You know the breakpoints, you generate five sizes, you are done. With four front ends on different frameworks, different devices and different release cycles, a fixed set is wrong for at least three of them within a quarter, and the process for adding a size is a cross-team negotiation.

    The second thing headless changes is who is holding the responsibility. In a monolith the platform knew both the layout and the asset. Split them and neither side knows both, so the contract between them has to carry enough information for the front end to decide.

    The API contract that works

    Return three things per asset, per product:

    A stable identifier. Opaque, permanent, meaningful only to the asset system. This is what the product record stores, for the reasons in wiring a DAM to a PIM.

    A canonical delivery URL. The base from which any rendition can be constructed by appending parameters. Not a rendition itself.

    Enough metadata to render responsibly. Intrinsic width and height, so the front end can reserve space and avoid layout shift. The default description, so alt text exists without another call. Focal point if you have one.

    That is the whole contract. Notably absent: a list of sizes. The moment the API returns thumbnail, medium and large, you have encoded three front ends’ assumptions into a shared interface, and the fourth front end will either misuse one of them or ask for a fourth name.

    Let the front end ask for what it needs

    With a URL-parameter delivery layer, each surface constructs what it wants at request time. A Cloudinary URL is the clearest example of this contract in practice, because the parameters sit in the path and can be generated by the consumer without a round trip.

    For a responsive web front end the useful set is narrow. Ask for automatic format so AVIF and WebP go to browsers that support them and JPEG to those that do not, which is what the format support matrix otherwise makes you handle by hand. Ask for automatic quality so compression is chosen per image. Then emit a srcset with the widths your layout actually uses, following the responsive images guidance, and let the browser choose.

    For a native app, ask for the device pixel ratio you have and nothing else. For a screen in a store, ask for the fixed size of the screen. Each surface answers a question only it can answer, and none of them need a deployment in another repository to do it.

    The performance argument for this is not subtle. Images are usually the largest contentful paint element on a product page, and serving a correctly sized modern format is most of the fix. Delivering WebP or better instead of an oversized JPEG typically moves that metric more than any front-end optimisation you will do this quarter.

    One source square linked by teal parameter lines to eight differently proportioned outputs, with the same eight redrawn in amber inside a hatched storage container

    Caching, which is the part that surprises people

    A derived-rendition model produces a very large number of distinct URLs, and that is fine, because each one is immutable.

    The rules that keep it fast:

    • Make the URL the cache key and make it deterministic. The same parameters must produce the same URL every time, in every front end. A shared helper is worth more than a documented convention.
    • Cache derived output aggressively and for a long time. An immutable URL can be cached effectively forever, following the standard HTTP caching guidance. The first request pays the transformation cost and nothing else does.
    • Version by identifier, not by cache busting. When an asset is replaced, the identifier or version component changes and the old URL simply stops being referenced. Purging a CDN by pattern is a thing you should never need to do routinely.

    The consumption implication matters for the bill as well. Repeated delivery of an already-generated derivative does not re-run the transformation, so the cost of the long tail of URLs is storage and bandwidth rather than compute.

    Do not put media logic in the front end

    The failure mode of a headless stack is that each front end grows its own image utility, and after a year the four utilities disagree.

    Keep a single shared definition of the shapes your business uses. Named transformations are the mechanism if you are on a platform that supports them: define product_card, product_hero, swatch once, and let every front end reference the name. When the design changes, the definition changes and every surface follows, without a coordinated release.

    If your delivery layer has no such feature, achieve the same thing with a small shared package that constructs URLs. The important property is that the parameters exist in exactly one place.

    Nine flat edge nodes ringing a teal-edged origin cube, three of them holding an amber cube that does not match the origin silhouette

    When is a headless media layer overkill?

    If you have one front end and no plans for a second, most of this page is theory. A monolithic storefront with a built-in media library, a sensible set of breakpoints and a CDN in front of it is a perfectly good architecture and will outperform a badly assembled headless stack.

    The threshold is the second surface. The moment a native app, a marketplace, a partner site or a physical screen consumes the same catalogue, the assumptions that made a fixed rendition set workable stop holding, and every month you delay makes the change larger.

    That is also the reason to put the delivery layer in early rather than after the second front end arrives. Adopting it on day one costs a URL helper and an account. Adopting it in month eighteen means finding every hard-coded image path in three codebases and every stored derivative in the catalogue. The developer-side version of that argument is in DAM for developers, and the properties an asset needs in order to be addressable this way are in what a DAM actually does. If you are also choosing a product API, what to demand from a PIM API before you buy covers the other half of the contract.

    Related field note

    Continue with How to measure headless commerce media.

    Related field note

    Continue with A practical checklist for headless commerce media.

  • Wiring a DAM to a PIM without a nightly sync job

    Wiring a DAM to a PIM without a nightly sync job

    The integration gets specified in one line on a whiteboard: “the PIM pulls images from the DAM overnight”. Six months later there is a queue, a staging directory, a retry script, a spreadsheet of failed transfers, and a rule that nobody publishes on Thursdays.

    The line on the whiteboard was the problem. Everything after it was consequence.

    Short answer: do not copy assets or asset metadata into the product system. Store a stable asset identifier on the product record and resolve it to a delivery URL at request time. The integration becomes a reference plus a URL contract, there is nothing to schedule, nothing to reconcile, and nothing that can be stale, because nothing was copied.

    Why the scheduled copy fails, specifically

    It is worth naming the failure modes rather than asserting that batch is bad, because plenty of batch integrations are fine and this particular one is not.

    It is stale by construction. Between two runs, the product system holds a claim about an asset that may no longer be true. If the window is twenty-four hours, your worst case is a day of serving a withdrawn image. That is a compliance question, not a latency question.

    Failures are silent and asynchronous. A run that half completes leaves the two systems in a state neither of them can describe. The person who finds out is a customer.

    It duplicates governance. Once the licence expiry date exists in both systems, one of them is wrong, and you now need a rule about which. That rule is the beginning of a reconciliation layer, and reconciliation layers grow.

    It cannot answer new questions. Six months on, a channel wants a 4:5 crop. The batch has been copying 1:1 and 16:9. Now you are changing the pipeline, backfilling, and re-running against the full catalogue.

    The general shape here is extract, transform, load, which is the correct tool for moving analytical data and the wrong one for keeping two operational systems in agreement about the same object.

    The reference contract

    Three pieces, and they are all small.

    One: the product record stores an identifier, not a file. A stable string that means “this asset” for the life of the asset. Not a URL, because URLs carry delivery parameters and those change. Not a filename, because filenames get edited. An identifier.

    Two: the asset system resolves an identifier plus a set of parameters into a delivery URL. This is the whole API surface you need for reads, and it is why the pattern works: the URL can be constructed by the consumer rather than requested from the provider. In Cloudinary the identifier is the public ID and the parameters sit in the URL path, so a storefront can build the exact URL it needs without a round trip to the asset system at all.

    Three: changes are pushed, not polled. When an asset is replaced, withdrawn or re-approved, the asset system emits an event and interested systems react. Webhook notifications are the standard mechanism. What you cache, you cache with an explicit invalidation path rather than a fixed interval.

    That is the entire integration. No staging directory, no queue you maintain, no Thursday rule.

    Two matte volumes joined by a single taut teal filament through a ring, with a slack amber cable discarded on the ground beneath

    What about the metadata the product system genuinely needs?

    This is the fair objection, and it has a fair answer.

    Some asset metadata really does need to be queryable from the product side. Merchandising wants to filter products whose primary image is missing. Legal wants a list of products using assets that expire this quarter. Neither of those can be answered if the asset metadata lives entirely elsewhere and is only resolvable one record at a time.

    The answer is a read model, not a copy. Maintain a projection of the few asset fields the product side needs to query, populated by events, explicitly labelled as derived, and never editable in the product system. Three fields is typical: asset exists, approval state, expiry date. That is a cache with a defined invalidation, which is exactly what the authority rule in DAM vs PIM permits.

    The discipline is that nobody may add a fourth field to the projection without saying which query needs it. Left ungoverned, a read model becomes a full copy in about a year, and you are back where you started.

    The crop problem, which is where most of the copying comes from

    Look at any batch integration between an asset library and a product catalogue and most of the volume is not masters. It is derivatives: the square one for the marketplace, the wide one for the hero, the small one for the cart.

    Those exist as files because at some point a system needed a shape and the only way to get it was to make one. If renditions are derived from the master by parameter, that entire class of transfer disappears. The product system stores one identifier, and each surface asks for the shape it needs at the point of use. Named transformations are worth using here rather than raw parameter strings, because they give each shape a name your storefront code can refer to, and let you change the definition later without editing the consumers.

    There is a governance caveat worth stating plainly, because vendor material tends to skate over it. Access control in most asset platforms, Cloudinary included, is organised at the folder level rather than per asset, and there is no general API for traversing arbitrary relationships between assets. Design the integration so it does not depend on either. Model the product-to-asset relationship on the product side, where it belongs anyway, and use folder structure to carry the permission boundaries you actually need.

    A timeline of twelve teal scheduled ticks with irregular pale change bars above, three ticks replaced by amber crosses leaving changes unpaired

    When is a scheduled export still correct?

    Three cases, and they are real.

    • The destination cannot make an outbound call. Some marketplace and retailer feeds accept a file on a schedule and nothing else. That is their contract and you meet it. Generate the feed from live data at generation time rather than from a copy you keep, and the rest of this page still applies upstream of it.
    • You are migrating. A one-time bulk move is a batch job by nature. Cloudinary’s migration tooling exists for exactly this, and it should terminate rather than become a schedule.
    • Analytical reporting. Copying into a warehouse for analysis is fine, because nothing operational reads from it.

    Everything else that is currently a schedule is a reference waiting to be written.

    How do you migrate an existing sync to references?

    Incrementally, and in this order. Add the identifier field to the product record and populate it from the existing copies, which you can usually do by matching on filename or hash. Switch one surface, ideally a low-traffic one, to resolve from the identifier. Add the event subscription so the read model stays current. Then stop the batch and delete the staging directory, which is the step teams postpone for a year because the old pipeline is still technically working.

    The API-side questions to ask the product platform before you commit to any of this are in what to demand from a PIM API before you buy, and the wider principle is in single source of truth applied to product data. If the four-system version of the boundary is still unclear, DAM, PIM, CMS and ERP maps it. For the developer-facing case for adopting a derived-rendition model from the first upload rather than after the pipeline exists, see DAM for developers.

    Related field note

    Continue with Asset-to-SKU mapping without duplicate files.