Introduction to MDM and PIM systems using Akeneo and Pimcore as an example

“As the enterprise grows, data chaos grows. Sometimes it starts with information about products (for example, from trading companies), but it can also start with customer data, with duplication of information about infrastructure (for telecom), etc. Sooner or later, the question of improving data quality (i.e. improving correctness, removing duplicates, and standardizing) arises.

At kt.team, we often advise clients on data quality, and it's time to talk about what our trading clients usually start with: PIM systems.”

MDM and PIM systems: what they are and why they are needed

Classic MDM-a system is a system that knows about various data sources and is responsible for data quality, that is, it contains”gold record” data. For example, your points of sale have one customer information, an online store has another, and marketing services have a third. The MDM system can bring all addresses and customer names to a single standard, find the same customers recorded differently, and eliminate errors based on different algorithms.

MDM systems manage different areas of data that are often referred to as domains (“customers” domain, “products” domain, etc.).

Today we can say that the PIM system is an MDM system under the “products” domain.

Thanks to MDM, a single space contains validators for changes, integrations with other systems, and rules for uploading this information to other systems.

All possible attributes, everything about each entity, everything is centered in MDM.

It is much easier to understand the essence of MDM systems if we consider PIM systems as an example. Therefore, we will talk about them later.

PIM systems allow you to concentrate all information from operators, photographers, suppliers, as well as various product rules. With the help of PIM systems, you can manage different product presentations on marketplaces and social networks.

The basic list of PIM system features:

• define all attributes for your products and their obligation for the channel (for example, a picture may be required for a website, but not for uploading to 1C);

• identify product categories from different sources;

• assess the quality of product data according to criteria of completeness and reliability using both simple validations and complex algorithms;

• set up algorithms for operators responsible for information about pictures. For example, tell them which products to fill in first. When posting an autumn-winter collection in August, you can set the main priority for sources to “own site”, then for “marketplace” sources, and then for other attributes and product categories;

• have uploads to all third-party sources such as ERP, WMS and CMS systems, distributors or even marketplaces (Wildberries, OZON, Yandex.Market, etc.) in one place with some pricing caveats;

• define a workflow for verifying product information (for example, for a brand manager to check the information before publication);

• for PIM systems with DAM, you can determine the workflow by files associated with the product (i.e., pictures and documents), as well as by meta-information about pictures (when the photo was taken, what type of shooting, whether this document is a certificate, what is its issue date and expiration date, etc.). );

• manage the marketing presentation of the product (on Instagram, the picture and description are the same, on the site — others, in advertising for the “mommy” segment — the third). As a rule, such systems are proudly referred to as PXM systems.

As a rule, PIM systems also have the ability to customize product statuses (“Validation by manager”, “Ready to publish”, etc.) and bulk product processing tools (quickly download something from an XLS file, update the general attribute for a product group, etc.).

The most advanced PIM systems include DAM-systems (or PLM-systems), i.e. file and picture management systems that may have their own attributes.

In this case, images that are product attributes may have their own attributes, for example, “Side image”, “Certification”. You do not just specify an image as an attribute, but this image itself is in a specific directory, can be reused, and a separate workflow can be created based on it (“retouched”, “outdated”, “needs to be reshot”).

When does a business need a PIM system

• If you have complex product business processes (for example, your own photo production or a large staff supporting information and content),

• if you have a lot of information about products (production, brand managers, sales — everyone generates their own XLS tables and is already juggling them),

• if you have a large number of information systems,

• if in current information systems like 1C your operators spend a lot of time on routine operations,

then you probably need an MDM/PIM system.

You will be able to simplify all business processes and relieve other sources of unnecessary information. For example, why should 1C store images for the site and the category in which this site is located?

Let's look at the two systems we're most expert in developing on: Akeneo and Pimcore. From the point of view of the technology stack, they are very similar: PHP/Symfony 4, MySQL, and Elasticsearch. But these are two very different systems in approach.

Pimcore MDM/PIM system

Pimcore is an MDM system. Despite the fact that its name sounds PIM, de facto it simply has a layout that allows you to work with products. There is also CRM (a layout that allows you to work with customers), plus you can create any other entities that are needed to solve business problems.

This does not mean that PIM functionality is weak there. On the contrary, it is very powerful, and it also has the ability to use deduplication.

I just want to emphasize that Pimcore was originally designed not only for product information management. This explains both the high complexity of configuring the system and its maximum versatility (both in terms of attributes and in terms of API for these entities).

Akeneo PIM system

Unlike the universal Pimcore, Akeneo is only a PIM system, i.e. a highly specialized tool.

There are two versions of this product: Enterprise and Community. Architecturally, this is one system (one core) with a different set of modules. The Enterprise version differs in that it has more modules and for the most part they are related to product management in large organizations (where there are many approvals, complex access rights, where operators have a large hierarchy, etc.).

A powerful attribute hierarchy, as in Pimcore, is missing here. Attributes are divided into attributes, attribute groups, and families. There is no way to create an “abstract entity” from which to inherit. For example, you can't create “Furniture” with your own attributes and then expand it with “Sofa” and “Table”, which will be added as “Furniture” attributes.

The main tasks of PIM systems

1. Allow all stakeholders to make the necessary changes and additions to product data in one area and ensure access to updated data in other areas.

Demo: searching et appending products in Akeneo PIM.

2. Resolve data conflicts that, in turn, can have a significant impact on doing business. This includes prioritizing sources, algorithms for acting in the presence of conflicting data, algorithms for standardizing information, etc.

3. Download data from various sources: CMS, CRM, ERP systems, highly specialized programs like CAD/CAD (for example, store all documentation, drawings and product properties in one place).

4. Upload data to different sources with control over the mandatory attributes. For example, if we want products without pictures not to appear on the site, we set this up in the PIM system. At the same time, we provide operators with transparent information about the products so that they can see which ones do not make it to the site and for what reason. This also includes uploading feeds for vendors or centrally sending uploads to marketplaces like Wildberries, OZON, Amazon, Yandex.Market, AliExpress, etc.

Demo: import and export data in Akeneo PIM.

5. Ensure the processing and replenishment of content in the system, taking into account the policy of rights and roles in the company. Any employee with the appropriate access rights, such as a marketer, sales manager, or even a copywriter, can add content.

As additional information about products, you can add descriptions, images, and video reviews. The text can be automatically translated into other languages within the system, which makes it much easier to adapt data to local requirements. If we deliver goods, for example, to Spain, it is better to give Spanish translators access to Spanish attributes and only for products that are shipped to this country.

Demo: roles in Akeneo PIM.

Akeneo and Pimcore for complex products

Let's say your business is cellular services. You have a tariff scale that has a lot of nuances for different regions of the country. In some regions, some tariffs are not applicable at all. In this case, pricing is a product property (price is not always a product property, more often a property of a sales channel, but this is where it is a product property).
Your product has complex pricing, and it is important for you to know and control all price factors.

Another example: you make sofas. Upholstery for them can be made from thousands of materials of different colors and textures. They differ in size, folding mechanism, armrest material. These differences are not just product properties, they depend on each other in a certain way: sofas in some price ranges can be upholstered only from a certain type of fabric and only in a limited number of colors, and some designs have different options.

Let's take a look at how Akeneo and Pimcore help you work with data for such complex products.

Akeneo supports a flat data model (with nuances: the Enterprise version may have composite attributes, i.e. an attribute can contain its own attributes).

Thus, such tasks boil down to the fact that Akeneo is expanded with a special configurator: developers write a module to manage a complex hierarchical structure in Akeneo's flat structure.

A composite attribute in Akeneo Enterprise and Pimcore means that the attribute contains its own attributes. For example, the “Color” attribute may contain a name, HEX and CMYK codes, and a picture, while the “Fabric” attribute may contain the name of the fabric and the conditions for its washing/washing.

In Pimcore, this task is solved by a boxed interface: any class can be extended, and that class, in turn, can also be extended. Attributes can be a relationship, including many-to-many, composite, and computed.

The most interesting cases for understanding the work of Akeneo and Pimcore may be as follows: a business has some basic products with a base price (for example, a basic sofa model upholstered in a certain fabric), and when you add a configuration, you need to automatically change the price (a sofa with a different upholstery costs more), and do it according to the rules (there are several categories of fabric, and the price for each next category increases by 100 rubles per linear meter).

The interface for configuring attributes in Akeneo looks like this:

At Pimcore:

Pimcore and Akeneo functionality

Pimcore is a flexible, convenient and multifunctional MDM system. Let's take a closer look at its features and discuss which of them Akeneo also has.

Data Quality/Semantics

Both Pimcore and Akeneo maintain connectivity and high quality for various data sets: structured and unstructured, internal and external. Information can be collated, verified and standardized to ensure that you always have up-to-date and accurate data for operations. For example, you can enrich product information using other websites and cloud services. From the point of view of data quality management, Pimcore is architecturally richer, but Akeneo is more visual.

These systems also allow you to format a variety of data sets based on industry standards, business rules, unique tests, metadata, and machine learning. You can change data values according to domain restrictions, data integrity restrictions, or other business rules.

Workflow management (workflow, validation workflow) and status model for products

Sometimes you need to navigate items by status just like a task tracker does a task. For example, the sequence of product statuses may be as follows: “New”, “Under review by a copywriter”, “Under review by a brand manager”, “Published”.

Both Akeneo (Enterprise) and Pimcore have such an opportunity.

It is possible that when selling goods between statuses, the brand manager will notice an inaccuracy in terms of attributes. Then he will be able to leave a comment and send the product for revision, and the operator will see the need for adjustments. Another option: sometimes you need to write a validator.

The status may be related to the launch of the import or export procedure for this product data.

Pimcore has the concept of validators and rules for canceling an operation. For example, you cannot publish a product if its price has not been updated for more than 30 days, you cannot submit a product with the “Blocked” status for validation, and so on.

These validations can be calculated (including sending a request to an external system for a decision).

The Pimcore workflow interface looks like this:

Digital asset management — DAM (aka product content management)

In any PIM system, even without DAM, it is possible to upload product images.

However, sometimes we need to make the files themselves independent entities (for example, photographers just take photos and don't specify the article number, or we need to reuse the same photo in different products). In this case, DAM is needed. It was first called PCM, and it seems the name comes from SAP. It's just DAM now.

DAM is a system for storing files (pictures, documents) with their own attributes. Sometimes we need not only to know the article photo, but also to distinguish that it is a side photo. It is not easy to see that this is a scan of the certificate, but also to determine when it expires. That is, the file itself is overgrown with metadata and this metadata can be associated with products (for example, you cannot publish an item with an expired certificate).

All of this functionality can be configured in DAM.

As a rule, DAM connects to where content is generated (so that they can use standard NFS or some other network access protocol to the “folder”), and content managers use DAM files in their products (assign photos and certificates to specific items).

However, if we need to send all the certificates, we can do it with one click (even provide a public download link so as not to send gigabytes by mail). And only certificates whose date is current will be uploaded.

DAM is part of both Pimcore and Akeneo (Enterprise only, although you can connect any open source DAM to the Community or even write your own if the requirements are modest).

Pimcore also has simple features for working with documents and editing images (through third-party libraries).

Versioning

Versioning shows when, by whom, and how exactly the product was updated (GIT for products).

It allows you to roll back to the previous version, submit a new version for approval and simply understand that if a product has updated one attribute that is important, for example, only for the site, then you only need to upload the update to the site, and not report the changes to the rest.

These features are available in Akeneo Community (although the functionality is greatly reduced here), Akeneo Enterprise, and Pimcore.

Data quality and attribute occupancy

It may be necessary to know that we do not have all the necessary attributes in the context of channels (the channel is the recipient of information: website, marketplace, ERP, etc.).

And sometimes you need to know this not just in the context of a channel, but also in the context of a product group. In Akeneo Enterprise, you can monitor the content and quality of group data by source — this is called Teamwork Assistant. For example: the autumn-winter season is coming soon, and we are separating all products for this season into a separate group. As a result, we see the occupancy rate across channels with the group filter applied, and it's easier for us to track progress in terms of occupancy.

Downloading product data from Excel and other spreadsheets

In addition to customizable import and export profiles, both systems (Akeneo and Pimcore) have slightly different approaches to data.

Out of the box, Pimcore allows you to work very flexibly with external tables directly from the interface, and for any entity (attributes, products, customers). You can upload any spreadsheet document directly into the interface and specify which fields to enter and where to enter.

Akeneo is designed to work with products and has an information enrichment service (Franklin), as well as fairly simple import and export mechanisms (you can download the table by setting up an import profile). Franklin collects product data online and cleverly integrates it, eliminating data inconsistencies.

Providing product information by external users (suppliers, manufacturers)

Sometimes it is necessary to make sure that product information is enriched at the manufacturer's expense.

I recall a food delivery service that has a large number of suppliers, all of whom must supply product information on their own. Internal users only confirm that this information has been downloaded and verify that it is correct.

Akeneo Enterprise has the Onboarder cloud service for this, while Akeneo Community and Pimcore have the only option to use existing access rights and flexibly configure them so that suppliers still have certain access.

However, it is better to configure the import of data from their sources (and most often some such imports already exist).

PIM and PXM, or a few words about marketers

PXM (product experience management) is a PIM that can deliver different information to different marketing channels.

While classic PIM considers product information as a “golden record” (yes, a product may have different localizations — language, etc. — but this is one set of attributes), PXM considers the product as a set of descriptions for different marketing channels.

Conventionally, if you produce water, you should position it in one way for moms, and in another way for offices. The product is the same, but its marketing is different.

Akeneo says that its Onboarder and Franklin, along with PIM itself, are PXM. In fact, both Akeneo and Pimcore provide a similar interface for real marketing activity: they are product variations that can be combined either by connections (Akeneo and Pimcore) or by inheritance (Pimcore).

EDI and PIM

If you receive EDI messages (for example from a European manufacturer) and want to automatically upload them to the information system, development is essential.

There is not much useful information there, but it is quite suitable for initially filling out information about the product (which will then go through complex enrichment workflows).

The reasons are both in the variety of EDI messages and their formats, and in the rare need for such exchanges.

However, in Russia, the demand for EDI is increasing, and the variety of formats is being smoothed out by large EDO centers (VLSI/Diadok/SKB Kontur, etc.), so this task is transformed into a task of integration with a local EDO provider.

MDM, PIM and microservice approach

The issue of attitude towards products is important if your company has a microservice ecosystem.

Strictly speaking, without Akeneo improvements, you won't be able to manage Akeneo PIM business processes from your BPMS. At Pimcore, things are slightly easier due to the rich capabilities of workflow access through the API. However, we recommend that you do not integrate such business processes and treat PIM as a closed system that has its own business processes. All you can do with such a system from other services is use its API.

Sometimes business processes and products are very intertwined. For example, when a product is the tariffs of one of the telecom operators that have complex business processes, but are extremely intertwined with promotional processes, regional time zones and billing. In this case, it is better to manage products through Camunda's business processes and other interfaces associated with Camunda.

Strictly speaking, the larger and more complex the processes are, the less suitable they are for modular systems (albeit open source).

Another option is to develop a module for Akeneo or Pimcore so that interfaces and validators are obtained from Camunda or JBPM and displayed by the PIM system. Of course, this is not a cheap development, but such a solution will allow maintaining the transparency of business processes from BPMS while maintaining a rich PIM interface.

Conclusion

So, we looked at the capabilities of the Akeneo and Pimcore systems with many of their similarities and differences. It's worth remembering that Akeneo is still a PIM system that is aimed at working with products, while Pimcore is a more extensive solution, an MDM system.

As for the cost of Akeneo, the Enterprise version will cost at least 30,000 euros per year, and redesigning Community can sometimes be cheaper.

For a better introduction to this topic, you may find these materials useful (all lines are clickable):

• ‍PIM system: how not to drown in information and keep everything under control;
• integration with the PIM system;
• integration with Pimcore;
• what are MDM-, PIM-, DAM-, PXM systems;
• ‍and the next video.

‍