Sitecore Experience Commerce – Creating a catalog

As there is not much online yet on how to create a catalog/product import in Sitecore Experience Commerce(XC) 9.0, I wanted to share a bit of code and thoughts on how we created it.

Where to start?

Since there was not much info about creating a catalog import, either in documentation or online, we had to gather our knowledge in another way.

We knew that XC contained an out-of-the-box catalog export/import, so we decided to have a look at the code through decompiling the assemblies. By doing this we got a couple of architectural insights:

• API controllers never run pipelines directly

• Minions can watch Lists

• There are Entity specific and generic pipelines, for example

  • IGetCatalogPipeline
  • IFindEntityPipeline

• Sitecore XC uses the Sitecore XP OData API to get items, like translations and settings

Next to that, another great help was digging through the tables in the two databases.

In this CatalogEntities table we can see each Entity has a couple of properties in the database:

• Id: Each Entity has its own unique Id, composed of an Entity Prefix followed by something unique
• EnvironmentId
• Version
• EntityVersion
• Published
• Entity: A JSON string containing all the details and properties of the Entity itself

These properties in the database together with the properties from the Entity JSON, give us a clear image on how to create these Entities, or at least which fields on the object you need to fill.

The creation flow

We want our catalog import to both handle the creation and update of categories and sellable items. This can be done by using the pipelines, minions and lists in Sitecore XC. So, what would this catalog import look like?

In above flowchart you can see an example flow on how to create the import. As you can see we use the ‘generic’ IFindEntityPipeline to get the entities if they exist. We use this pipeline and not the GetCatalogPipeline for example because the latter one also gathers a lot of data which we don’t need in these imports/updates, like pricing and inventory details.

Now we want to make sure that we can also update the products from time to time, but don’t want to run a full update/import command. If only one product has changed in the PIM, we only want to update that single product. To make this possible we add the Id’s of the products to a List and process those Id’s using a Minion.

This Minion would then run another pipeline to update the product. Using a List here makes scaling the import process also possible, because you can spin-up multiple Environments/Minions to process the Id’s.

The actual code

I am not going to place all the code for this import inside this blog post, instead I created a GitHub repository to host an example implementation: https://github.com/GuidovTricht/SitecoreCommerceProductImportExample (Note that this example doesn’t contain the List and Minion to create the SellableItems)

But I do want to point out some steps and code snippets. For example, on how to create an Entity. In the following snippet creating a Category.

public override async Task<SynchronizeCategoryArgument> Run(SynchronizeCategoryArgument arg, CommercePipelineExecutionContext context)
{
    Sitecore.Commerce.Plugin.Catalog.Category category = null;
    var categoryId = $"{CommerceEntity.IdPrefix<Sitecore.Commerce.Plugin.Catalog.Category>()}{arg.Catalog.Name}-{arg.ImportCategory.CategoryId.ProposeValidId()}";

    if (await _doesEntityExistPipeline.Run(
        new FindEntityArgument(typeof(Sitecore.Commerce.Plugin.Catalog.Category), categoryId),
        context.CommerceContext.GetPipelineContextOptions()))
    {
        category = await _findEntityPipeline.Run(new FindEntityArgument(typeof(Sitecore.Commerce.Plugin.Catalog.Category), categoryId), context.CommerceContext.GetPipelineContextOptions()) as Sitecore.Commerce.Plugin.Catalog.Category;
    }
    else
    {
        var createResult = await _createCategoryPipeline.Run(
            new CreateCategoryArgument(arg.Catalog.Id, arg.ImportCategory.CategoryId.ProposeValidId(),
                arg.ImportCategory.CategoryName, ""), context.CommerceContext.GetPipelineContextOptions());
        category = createResult?.Categories?.FirstOrDefault(c => c.Id.Equals(categoryId));
    }

    Condition.Requires<Sitecore.Commerce.Plugin.Catalog.Category>(category)
        .IsNotNull($"{this.Name}: The Category could not be created.");

    arg.Category = category;

    return arg;
}

Notice that we used a couple of standard Helpers/Extensions like CommerceEntity.IdPrefix<>(), which returns the default prefix like we have seen in the database, and ProposeValidId(), which removes any non alphanumeric characters from the (id)string.

In the following snippet you can see how easy it is to add a list/enumerable of strings to an XC List:

List<string> ids;
var addArg = new ListEntitiesArgument(ids, "ProductUpdatesList");
await _addListEntitiesPipeline.Run(addArg, context);

And in the last snippet what a Minion would look like which reads 10 Id’s from the List each time it is run and then uses it to run a pipeline.

public class UpdateProductsMinion : Minion
{
    private ISynchronizeProductPipeline _synchronizeProductPipeline;
    private IRemoveListEntitiesPipeline _removeListEntitiesPipeline;

    public override void Initialize(IServiceProvider serviceProvider, ILogger logger, MinionPolicy policy, CommerceEnvironment environment, CommerceContext globalContext)
    {
        base.Initialize(serviceProvider, logger, policy, environment, globalContext);
        _synchronizeProductPipeline = this.ResolvePipeline<ISynchronizeProductPipeline>();
        _removeListEntitiesPipeline = this.ResolvePipeline<IRemoveListEntitiesPipeline>();
    }

    public override async Task<MinionRunResultsModel> Run()
    {
        var listCount = await this.GetListCount(this.Policy.ListToWatch);

        this.Logger.LogInformation($"{this.Name}-Review List {this.Policy.ListToWatch}: Count:{listCount}");
        var list = (await this.GetListIds<string>(this.Policy.ListToWatch, this.Policy.ItemsPerBatch)).IdList;

        foreach (var id in list)
        {
            this.Logger.LogDebug($"{this.Name}-Reviewing Pending Product Update: {id}");

            try
            {
                await _synchronizeProductPipeline.Run(
                    new SynchronizeProductArgument { ExternalProductId = id },
                    MinionContext.GetPipelineContextOptions());
                this.Logger.LogInformation($"{this.Name}: Product with id {id} has been updated");

                await _removeListEntitiesPipeline.Run(new ListEntitiesArgument(new List<string>(){ id }, this.Policy.ListToWatch), MinionContext.GetPipelineContextOptions());
            }
            catch (Exception ex)
            {
                this.Logger.LogError(ex, $"{this.Name}: Product with id {id} could not be updated");
            }
        }

        return new MinionRunResultsModel
        {
            ItemsProcessed = this.Policy.ItemsPerBatch,
            HasMoreItems = listCount > this.Policy.ItemsPerBatch
        };
    }
}