Tag: Architecture

How the pieces fit and which properties still hold after a replatform on either side of the boundary.

  • Build versus buy for the product catalogue

    Build versus buy for the product catalogue

    An engineering team looks at a product platform quote, looks at the feature list, and concludes correctly that they could build eighty per cent of it in a quarter. They are right. The trouble is the shape of the remaining twenty per cent, which is not eighty per cent smaller.

    Short answer: building a product catalogue is defensible when your model is unusual and your channel count is low, and it is a permanent commitment rather than a quarter. Building a media delivery layer is almost never worth it, because the hard parts are format negotiation, caching and edge delivery, all of which are commodity and none of which are your product. Split the two decisions.

    What you are actually buying

    Strip the marketing and a product platform sells four things.

    A modelling engine that can express families, products, variants, typed attributes and localised values, and enforce them. This is the part teams underestimate, because a first version is easy and the fifth revision is not.

    A completeness engine that scores records against per-channel rule sets and blocks publication. This is the part that pays for itself.

    A workflow layer with roles, states and assignment. This is the part most teams already have somewhere else and often do not need twice.

    Channel mappings to marketplaces and retailers, maintained by someone else as those specifications change. This is the part with no ceiling, because the specifications change without asking you.

    You can build the first three. The fourth is a subscription to someone else’s maintenance burden, and it is often the honest reason to buy.

    The build case, stated fairly

    There are real conditions under which building is correct.

    Your model is genuinely unusual. Configurable products, made-to-order dimensions, service bundles, regulated goods with jurisdiction-specific field sets. If every platform demo ends with “we would handle that with a custom field”, building is worth costing properly, because a platform you have to fight is worse than no platform.

    You have one or two channels and no plans for more. The channel mapping burden is what makes building expensive over time, and if you only ever publish to your own storefront, that burden is small and stable.

    Product data is your product. Marketplaces, aggregators and comparison sites are in this position. Buying a generic catalogue to run a catalogue business is odd.

    You have engineering capacity that is not otherwise committed. Not “we could find the time”. Committed capacity, permanently, because this becomes a system with an on-call rota.

    If two or more of those are true, build is a real option. If none is, the discussion is usually about wanting to build rather than about needing to.

    What the maintenance actually costs

    The build estimate is nearly always for version one and the cost is nearly all after it.

    • Schema evolution. Every new attribute type, every new market, every change to what localisable means. Small individually, continuous forever.
    • Channel specification drift. Every marketplace changes its requirements at least annually, sometimes silently. Each change is a mapping update and a validation update, discovered through rejections.
    • Import and export tooling. Suppliers send spreadsheets in whatever shape they like. Ingestion is a long tail that never ends.
    • The interface. Cataloguers need one, and it is not a weekend project. This is the single most underestimated line in every build estimate.
    • Concurrency and audit. Who changed what, when, and what did it look like before. Easy to skip in version one and painful to retrofit.

    Cost the second and third years, not the first. A build that flattens after eighteen months is a good build; most do not flatten, and the curve to compare against is the total cost of ownership of the alternative rather than its licence line, as set out in what a DAM and a PIM actually cost.

    A central card ringed by three dashed circles holding six teal blocks, nine pale blocks and fourteen amber blocks spilling past the outer ring

    The media layer is a different decision

    Here the answer is much less balanced, and it is worth being direct about why.

    Building a media layer means owning storage, format negotiation, resizing quality, caching, invalidation and edge delivery. Each of those is a solved commodity problem with a long tail of correctness issues, and none of them differentiates your business.

    Format negotiation alone is a good example. Serving AVIF or WebP to browsers that support them and JPEG to those that do not, with correct Accept handling and cache keys that do not collide, is more subtle than it looks and has to be revisited as format support changes. Getting it slightly wrong produces either broken images for some users or a cache that serves the wrong format to the wrong browser, which is a bug that reproduces for nobody on the team.

    The realistic build here is object storage plus a resizing service plus a CDN, which is three moving parts, an ongoing cost, and no metadata layer at all. That last omission is the important one: you have built delivery, not asset management. Nothing in it knows what an asset depicts, what is permitted, or which version is current.

    The managed alternative starts free and stays cheap at small scale. Cloudinary’s Programmable Media free tier is 25 credits a month, where a credit is 1,000 transformations, 1 GB of storage or 1 GB of delivery. For a small catalogue that is not a trial, it is the whole requirement, and it comes with the structured metadata layer that the build-it-yourself version does not have.

    The case for adopting it on day one rather than after you have built the three-part version is simply that the migration is the expensive part. Starting with a master and a URL costs nothing and gives you the structure from the first upload. Arriving at it eighteen months later means finding every stored derivative and every hard-coded path in every codebase, which is the argument made in full in DAM for developers.

    Macro of a stack of printed sheets along the cut edge, orderly for two thirds and fanning into dog-eared amber sheets at the top

    The middle path most teams should take

    Buy the media layer, build the thin catalogue, and keep the option open.

    Concretely: put assets on a managed delivery platform immediately. Keep product data in whatever you have, modelled properly, with typed attributes and a real variant structure. Write the channel mappings as small, declarative, testable code rather than as a platform configuration. That combination costs almost nothing, and it produces a data model that can be loaded into a bought platform later without a cleanup pass, which is the expensive half of any migration.

    The decision point comes when the channel mapping work starts consuming a person. That is the moment the subscription to someone else’s maintenance burden starts looking cheap, and it is a much clearer signal than any threshold based on catalogue size.

    How do you decide this week?

    Score four questions honestly. Is the model unusual, is the channel count above two, is engineering capacity actually committed, and is product data your product. Two or more yeses point to build, and fewer than two point to buy.

    Then apply the same four questions to the media layer separately, and notice that the answers are almost always different. If they are not, one of the answers is probably wishful.

    The threshold test for whether you need a bought catalogue at all is in do you need a PIM yet, the API questions that determine whether a bought one will integrate cleanly are in what to demand from a PIM API before you buy, and the lighter-weight options that are neither building nor buying are in alternatives to buying a DAM and a PIM.

    Related field note

    Continue with The build-versus-buy decision for ERP product identifiers.

    Related field note

    Continue with The build-versus-buy decision for catalog launch readiness.

  • Single source of truth, applied to product data

    Single source of truth, applied to product data

    Every architecture document contains the phrase. Every architecture built from those documents contains the same product title in four places, three of which are editable.

    The phrase is not wrong. It is just not a design until it names fields.

    Short answer: single source of truth means, for each field, exactly one system where it can be edited. Everything else holds a reference, or a clearly labelled read-only cache with a defined refresh. Written that way it is enforceable. Written as a principle it is a sentence everyone agrees with while building the opposite.

    Write it as a table of fields, not as a principle

    The artefact you need is unglamorous: a list of every field that appears in more than one system, with the system that owns it, and the systems that consume it. Fifty to two hundred rows for most catalogues.

    Producing it takes about two days and it is the highest-value two days in any integration project, because it converts an architectural opinion into a set of decisions people can disagree with specifically. Vague disagreement recurs forever. Specific disagreement gets resolved once.

    The rules that make the list decidable are in DAM vs PIM for two systems and the four-system boundary map when the CMS and ERP are involved. Single source of truth as a concept is old and stable; the work is entirely in the application.

    What a legitimate copy looks like

    Absolutism here is unhelpful, because some copying is necessary. A storefront that resolves every field from an upstream API on every page load is slow and fragile. The distinction is not copy versus no copy. It is governed copy versus ungoverned copy.

    A legitimate copy has four properties:

    It is read-only in the consuming system. Not by convention. Enforced, with the edit control absent from the interface. If a person can edit it, someone eventually will, and their edit will be silently overwritten or silently authoritative, and you will not know which.

    It has a defined refresh path. Either an event that invalidates it or an interval that bounds its staleness. “It gets updated when we run the job” is a schedule; “it is refreshed within a minute of a change event” is a contract.

    It is labelled as derived. In the schema, in the interface, and in the documentation. The next person to look at it should not have to work out whether it is authoritative.

    It exists because a specific query needs it. Not because it was convenient to have. The discipline of naming the query is what stops a read model growing into a full duplicate over a year.

    Anything with all four is a cache. Anything missing one is a second source of truth wearing a disguise.

    One pedestal holding a teal-edged plate with five dishes of translucent replicas around it, two of them solid amber and mismatched

    The three fields that always cause trouble

    Product title. Owned by the product record, and immediately wanted by the CMS for page titles, by the DAM for asset naming, and by search indexes. The trap is the CMS letting an editor “adjust it for this page”, which produces a per-page override that is invisible from the product side. If overrides are legitimate, model them as an override field with the original still visible. If they are not, remove the edit control.

    Price. Usually originates in the ERP or a pricing engine, is consumed everywhere, and is the field where staleness is most expensive. It should never be copied without a short and enforced refresh, and it is the strongest case in the catalogue for pushing changes rather than polling for them.

    Image association. Which images represent this product. Owned by the product record. Copied into the CMS as a page-level selection, which is legitimate if the CMS choice is understood as “which of the product’s images to place here” rather than as an independent list. When it becomes an independent list, images that were withdrawn upstream keep appearing on pages, which is the failure that reaches customers.

    Reference resolution beats field replication

    The pattern that keeps the list short is to store an identifier and resolve it, rather than storing the values.

    A product record holding an asset identifier is one field. The same product record holding the asset’s URL, dimensions, alt text, rights status and expiry date is five fields, all of which can go stale independently, and each of which is a row on your ownership list.

    The same applies in the other direction. A CMS page that stores a product identifier and resolves the title, price and images at render time has one field to govern. A page that stores copies of all three has three.

    The reason this is not obvious is that resolution has a cost, and the cost is paid at request time in a system somebody is measuring. It is worth paying, and where it is not, the answer is the governed cache above rather than an ungoverned copy. The mechanics are in wiring a DAM to a PIM, which works through the same decision for the asset side specifically.

    A schematic of one source feeding three caches with different refresh intervals, teal one-way arrows, and a struck-through amber reverse line

    How the list decays, and what to do about it

    The ownership list is accurate on the day it is signed and drifts from then on. Three things cause the drift.

    A new system arrives and nobody adds its fields. Usually a marketing tool, a review platform or an analytics product that turns out to be writing back.

    A migration inverts an ownership. A replatform moves a field’s home and the list still describes the old arrangement. This is the most dangerous kind, because integrations built after the migration will follow the code rather than the document.

    A shortcut becomes permanent. Someone enables editing in a consuming system to unblock a launch, and it is still enabled two years later.

    The countermeasure is small and boring. Review the list when any system in it is replaced, and once a year otherwise. Put the review on the same calendar as something that already happens. A list nobody reviews is a historical document, and the first time it misleads someone it will cost more than the review would have.

    What is the smallest useful version of this?

    Ten rows, on one page, covering the fields that appear in more than two systems. Title, price, description, images, category, stock, identifier, rights, approval state, and whichever tenth field your team already argues about.

    Write the owner. Write the consumers. Write, for each consumer, whether it references or caches, and if it caches, what invalidates it. Get the owner of each system to sign the page.

    That page is worth more than the diagram it will eventually be drawn as, and it is the input to what to demand from a PIM API before you buy, because an API that cannot resolve references cheaply will force you into caching decisions you did not want to make. The modelling groundwork underneath it is in the product data model that survives growth.

  • Media in a headless commerce stack

    Media in a headless commerce stack

    The headless migration goes well. The catalogue is an API, the storefront is a separate application, a second front end appears for the app, and a third for the in-store screens. Then someone notices the app is downloading two megabyte hero images to a five inch screen, and the fix is a ticket in three repositories.

    Short answer: in a headless stack the product API should return an asset identifier and a canonical delivery URL, never a set of pre-baked sizes. Each front end appends the parameters it needs at request time. That keeps the shape decision in the front end that knows the viewport, and the asset decision in the system that owns the asset, which is the only split that stays stable as you add channels.

    What headless changes about media

    Headless architecture removes the assumption that there is one front end. That assumption was carrying more weight than anyone realised, because it was also the assumption that let you decide image sizes once, centrally, at build time.

    With one front end, a fixed set of renditions is workable. You know the breakpoints, you generate five sizes, you are done. With four front ends on different frameworks, different devices and different release cycles, a fixed set is wrong for at least three of them within a quarter, and the process for adding a size is a cross-team negotiation.

    The second thing headless changes is who is holding the responsibility. In a monolith the platform knew both the layout and the asset. Split them and neither side knows both, so the contract between them has to carry enough information for the front end to decide.

    The API contract that works

    Return three things per asset, per product:

    A stable identifier. Opaque, permanent, meaningful only to the asset system. This is what the product record stores, for the reasons in wiring a DAM to a PIM.

    A canonical delivery URL. The base from which any rendition can be constructed by appending parameters. Not a rendition itself.

    Enough metadata to render responsibly. Intrinsic width and height, so the front end can reserve space and avoid layout shift. The default description, so alt text exists without another call. Focal point if you have one.

    That is the whole contract. Notably absent: a list of sizes. The moment the API returns thumbnail, medium and large, you have encoded three front ends’ assumptions into a shared interface, and the fourth front end will either misuse one of them or ask for a fourth name.

    Let the front end ask for what it needs

    With a URL-parameter delivery layer, each surface constructs what it wants at request time. A Cloudinary URL is the clearest example of this contract in practice, because the parameters sit in the path and can be generated by the consumer without a round trip.

    For a responsive web front end the useful set is narrow. Ask for automatic format so AVIF and WebP go to browsers that support them and JPEG to those that do not, which is what the format support matrix otherwise makes you handle by hand. Ask for automatic quality so compression is chosen per image. Then emit a srcset with the widths your layout actually uses, following the responsive images guidance, and let the browser choose.

    For a native app, ask for the device pixel ratio you have and nothing else. For a screen in a store, ask for the fixed size of the screen. Each surface answers a question only it can answer, and none of them need a deployment in another repository to do it.

    The performance argument for this is not subtle. Images are usually the largest contentful paint element on a product page, and serving a correctly sized modern format is most of the fix. Delivering WebP or better instead of an oversized JPEG typically moves that metric more than any front-end optimisation you will do this quarter.

    One source square linked by teal parameter lines to eight differently proportioned outputs, with the same eight redrawn in amber inside a hatched storage container

    Caching, which is the part that surprises people

    A derived-rendition model produces a very large number of distinct URLs, and that is fine, because each one is immutable.

    The rules that keep it fast:

    • Make the URL the cache key and make it deterministic. The same parameters must produce the same URL every time, in every front end. A shared helper is worth more than a documented convention.
    • Cache derived output aggressively and for a long time. An immutable URL can be cached effectively forever, following the standard HTTP caching guidance. The first request pays the transformation cost and nothing else does.
    • Version by identifier, not by cache busting. When an asset is replaced, the identifier or version component changes and the old URL simply stops being referenced. Purging a CDN by pattern is a thing you should never need to do routinely.

    The consumption implication matters for the bill as well. Repeated delivery of an already-generated derivative does not re-run the transformation, so the cost of the long tail of URLs is storage and bandwidth rather than compute.

    Do not put media logic in the front end

    The failure mode of a headless stack is that each front end grows its own image utility, and after a year the four utilities disagree.

    Keep a single shared definition of the shapes your business uses. Named transformations are the mechanism if you are on a platform that supports them: define product_card, product_hero, swatch once, and let every front end reference the name. When the design changes, the definition changes and every surface follows, without a coordinated release.

    If your delivery layer has no such feature, achieve the same thing with a small shared package that constructs URLs. The important property is that the parameters exist in exactly one place.

    Nine flat edge nodes ringing a teal-edged origin cube, three of them holding an amber cube that does not match the origin silhouette

    When is a headless media layer overkill?

    If you have one front end and no plans for a second, most of this page is theory. A monolithic storefront with a built-in media library, a sensible set of breakpoints and a CDN in front of it is a perfectly good architecture and will outperform a badly assembled headless stack.

    The threshold is the second surface. The moment a native app, a marketplace, a partner site or a physical screen consumes the same catalogue, the assumptions that made a fixed rendition set workable stop holding, and every month you delay makes the change larger.

    That is also the reason to put the delivery layer in early rather than after the second front end arrives. Adopting it on day one costs a URL helper and an account. Adopting it in month eighteen means finding every hard-coded image path in three codebases and every stored derivative in the catalogue. The developer-side version of that argument is in DAM for developers, and the properties an asset needs in order to be addressable this way are in what a DAM actually does. If you are also choosing a product API, what to demand from a PIM API before you buy covers the other half of the contract.

    Related field note

    Continue with How to measure headless commerce media.

    Related field note

    Continue with A practical checklist for headless commerce media.

  • DAM, PIM, CMS and ERP: the boundary map

    DAM, PIM, CMS and ERP: the boundary map

    The vendor comparison pages are all written from inside one of the four categories, which is why they all conclude that the category they are written from should be the centre of your architecture. Read four of them and you will have four incompatible architectures, each internally consistent.

    The map below is written from outside all four.

    Short answer: the DAM owns the asset, the PIM owns the product record, the CMS owns the page, and the ERP owns the transaction. For any disputed field, ask which of those four things the field describes. That resolves most of them outright, and the leftovers are worth arguing about properly because they are genuinely contested.

    What each one is for

    DAM. The visual and rich media object, and everything intrinsic to it. What the asset depicts, who made it, what rights attach, which version is current, how it renders for a destination. Digital asset management as a category covers more than this in practice, but the intrinsic-to-the-asset test is the useful one. The longer version is in what a DAM actually does.

    PIM. Structured product information. Identifiers, attributes, variant structure, category placement, per-market copy, and channel completeness rules. Product information management exists because product data has hundreds of attributes across dozens of markets and a supply chain of contributors, and nothing else handles that shape well. See what a PIM actually does.

    CMS. Page and component structure, and editorial content that is not about a product. Which components sit on which page, in what order, with what copy, published when. A content management system is a composition tool. Its record is the page, not the things the page references.

    ERP. The transactional record. Stock, cost, purchase orders, suppliers, fulfilment, financials. Enterprise resource planning is upstream of everything above and is usually the system nobody in a marketing conversation wants to talk about, right up until someone asks where the cost price comes from.

    The field map

    Field by field, here is where authority usually lands.

    The DAM owns: master files and renditions, what an asset depicts, creator and shoot data, licence terms and expiry, releases, approval state, colour profile, asset version history, default image description.

    The PIM owns: SKU and GTIN, attributes and specifications, units, variant structure, commerce category placement, per-market product copy, channel rules, and the product-to-asset association.

    The CMS owns: page and component structure, navigation, editorial copy that is not product copy, publication scheduling for pages, and the choice of which asset appears in which slot.

    The ERP owns: stock levels, cost price, supplier and purchase data, fulfilment state, and the canonical existence of a product as a thing the business transacts in.

    Two of those deserve emphasis because they get placed wrong most often. The product-to-asset association belongs in the PIM, not the DAM, because it is a fact about the product rather than about the photograph. The canonical product identity usually originates in the ERP, not the PIM, because a product exists as a purchasable thing before anyone writes marketing copy about it. Getting the second one backwards produces two competing definitions of what a product is, which is the problem master data management exists to solve and which you would rather not need.

    A responsibility matrix on card, teal discs for ownership and hollow rings for consumption, three cells holding two discs and marked amber

    The two genuinely contested boundaries

    Most of the map is uncontroversial once written down. Two areas are not, and pretending otherwise wastes a lot of workshop time.

    Alt text and image descriptions. The DAM’s claim is that the text describes the image and should be reusable everywhere the image appears. The CMS’s claim is that good alt text is contextual, and the same photograph legitimately needs different text on a category tile, a size guide and a press page. Both are correct.

    The workable answer is two fields rather than one argument. The DAM holds a default description of what is depicted, which is intrinsic. The placing surface may override it, which is contextual, and the override is the exception rather than the norm. MDN’s guidance on the img element and the WCAG overview both support that reading: alt text serves the function of the image in context, and the context is not always known at asset level.

    Product copy in the CMS. Marketing wants long-form product storytelling built as page components, with layout. The PIM wants product copy as structured fields so it can be syndicated. Both are legitimate needs and they are not the same content.

    The rule that holds: copy that has to go to a channel other than your own site belongs in the PIM as a field. Copy that only ever appears on your own site, as part of a designed page, belongs in the CMS as a component. If a piece of copy has to do both, it is a PIM field and the CMS composes around it.

    How many of these do you need?

    Fewer than four, for most people, and the answer changes with catalogue shape rather than with company size.

    • Under a few hundred products, simple attributes, one market. The CMS or the storefront platform can carry product data. Skip the PIM until attribute count or locale count makes it painful, which is usually somewhere around a thousand SKUs or three markets.
    • One brand, one market, no third-party rights. The CMS media library may genuinely be enough and adding a DAM adds an integration without adding an answer.
    • Any real inventory or fulfilment. You have an ERP whether you call it that or not, and its data will be authoritative for the transactional fields regardless of what your architecture diagram says.
    • Multi-market, multi-channel, seasonal reshoots. You need all four and you need the boundary written down before any of them is bought.

    What you should not do is buy a system to solve a problem another one owns. A DAM bought to fix product data will not fix product data. A PIM bought to manage images will manage them badly, in a schema designed for attributes. A CMS asked to be the product master becomes the thing you replatform away from. DAM vs PIM sets out the two-system version of the same decision.

    Four filing volumes in a ring joined by teal bridges, with a single unsupported amber block floating between two of the bridges

    What do you do when two systems both offer the same feature?

    Three questions, in order.

    1. Which system holds the record this field describes? Licence expiry describes an image. Net weight describes a product. Component order describes a page. Stock describes a transaction. Most fields stop here.
    2. Which system’s users maintain it? A field maintained by cataloguers, placed in a system cataloguers never open, will be empty forever. Where question one leaves room, authority should follow the workflow.
    3. Which system can enforce it? Controlled vocabulary and validation in one, free text in the other, is not a real choice. Enforcement wins.

    Write the answers into an ownership matrix and get it signed by the owner of each system. That single artefact prevents more integration rework than any other document in the programme, and it becomes the input to the integration design in wiring a DAM to a PIM and the operating discipline in single source of truth applied to product data.

  • Wiring a DAM to a PIM without a nightly sync job

    Wiring a DAM to a PIM without a nightly sync job

    The integration gets specified in one line on a whiteboard: “the PIM pulls images from the DAM overnight”. Six months later there is a queue, a staging directory, a retry script, a spreadsheet of failed transfers, and a rule that nobody publishes on Thursdays.

    The line on the whiteboard was the problem. Everything after it was consequence.

    Short answer: do not copy assets or asset metadata into the product system. Store a stable asset identifier on the product record and resolve it to a delivery URL at request time. The integration becomes a reference plus a URL contract, there is nothing to schedule, nothing to reconcile, and nothing that can be stale, because nothing was copied.

    Why the scheduled copy fails, specifically

    It is worth naming the failure modes rather than asserting that batch is bad, because plenty of batch integrations are fine and this particular one is not.

    It is stale by construction. Between two runs, the product system holds a claim about an asset that may no longer be true. If the window is twenty-four hours, your worst case is a day of serving a withdrawn image. That is a compliance question, not a latency question.

    Failures are silent and asynchronous. A run that half completes leaves the two systems in a state neither of them can describe. The person who finds out is a customer.

    It duplicates governance. Once the licence expiry date exists in both systems, one of them is wrong, and you now need a rule about which. That rule is the beginning of a reconciliation layer, and reconciliation layers grow.

    It cannot answer new questions. Six months on, a channel wants a 4:5 crop. The batch has been copying 1:1 and 16:9. Now you are changing the pipeline, backfilling, and re-running against the full catalogue.

    The general shape here is extract, transform, load, which is the correct tool for moving analytical data and the wrong one for keeping two operational systems in agreement about the same object.

    The reference contract

    Three pieces, and they are all small.

    One: the product record stores an identifier, not a file. A stable string that means “this asset” for the life of the asset. Not a URL, because URLs carry delivery parameters and those change. Not a filename, because filenames get edited. An identifier.

    Two: the asset system resolves an identifier plus a set of parameters into a delivery URL. This is the whole API surface you need for reads, and it is why the pattern works: the URL can be constructed by the consumer rather than requested from the provider. In Cloudinary the identifier is the public ID and the parameters sit in the URL path, so a storefront can build the exact URL it needs without a round trip to the asset system at all.

    Three: changes are pushed, not polled. When an asset is replaced, withdrawn or re-approved, the asset system emits an event and interested systems react. Webhook notifications are the standard mechanism. What you cache, you cache with an explicit invalidation path rather than a fixed interval.

    That is the entire integration. No staging directory, no queue you maintain, no Thursday rule.

    Two matte volumes joined by a single taut teal filament through a ring, with a slack amber cable discarded on the ground beneath

    What about the metadata the product system genuinely needs?

    This is the fair objection, and it has a fair answer.

    Some asset metadata really does need to be queryable from the product side. Merchandising wants to filter products whose primary image is missing. Legal wants a list of products using assets that expire this quarter. Neither of those can be answered if the asset metadata lives entirely elsewhere and is only resolvable one record at a time.

    The answer is a read model, not a copy. Maintain a projection of the few asset fields the product side needs to query, populated by events, explicitly labelled as derived, and never editable in the product system. Three fields is typical: asset exists, approval state, expiry date. That is a cache with a defined invalidation, which is exactly what the authority rule in DAM vs PIM permits.

    The discipline is that nobody may add a fourth field to the projection without saying which query needs it. Left ungoverned, a read model becomes a full copy in about a year, and you are back where you started.

    The crop problem, which is where most of the copying comes from

    Look at any batch integration between an asset library and a product catalogue and most of the volume is not masters. It is derivatives: the square one for the marketplace, the wide one for the hero, the small one for the cart.

    Those exist as files because at some point a system needed a shape and the only way to get it was to make one. If renditions are derived from the master by parameter, that entire class of transfer disappears. The product system stores one identifier, and each surface asks for the shape it needs at the point of use. Named transformations are worth using here rather than raw parameter strings, because they give each shape a name your storefront code can refer to, and let you change the definition later without editing the consumers.

    There is a governance caveat worth stating plainly, because vendor material tends to skate over it. Access control in most asset platforms, Cloudinary included, is organised at the folder level rather than per asset, and there is no general API for traversing arbitrary relationships between assets. Design the integration so it does not depend on either. Model the product-to-asset relationship on the product side, where it belongs anyway, and use folder structure to carry the permission boundaries you actually need.

    A timeline of twelve teal scheduled ticks with irregular pale change bars above, three ticks replaced by amber crosses leaving changes unpaired

    When is a scheduled export still correct?

    Three cases, and they are real.

    • The destination cannot make an outbound call. Some marketplace and retailer feeds accept a file on a schedule and nothing else. That is their contract and you meet it. Generate the feed from live data at generation time rather than from a copy you keep, and the rest of this page still applies upstream of it.
    • You are migrating. A one-time bulk move is a batch job by nature. Cloudinary’s migration tooling exists for exactly this, and it should terminate rather than become a schedule.
    • Analytical reporting. Copying into a warehouse for analysis is fine, because nothing operational reads from it.

    Everything else that is currently a schedule is a reference waiting to be written.

    How do you migrate an existing sync to references?

    Incrementally, and in this order. Add the identifier field to the product record and populate it from the existing copies, which you can usually do by matching on filename or hash. Switch one surface, ideally a low-traffic one, to resolve from the identifier. Add the event subscription so the read model stays current. Then stop the batch and delete the staging directory, which is the step teams postpone for a year because the old pipeline is still technically working.

    The API-side questions to ask the product platform before you commit to any of this are in what to demand from a PIM API before you buy, and the wider principle is in single source of truth applied to product data. If the four-system version of the boundary is still unclear, DAM, PIM, CMS and ERP maps it. For the developer-facing case for adopting a derived-rendition model from the first upload rather than after the pipeline exists, see DAM for developers.

    Related field note

    Continue with Asset-to-SKU mapping without duplicate files.