Posts Tagged ‘Catalog’

For past few weeks, I am seeing few architecture questions in MSDN communities on how to mold catalog system to address client requirements. Catalog system is heart of the ecommerce solution and any architecture flaw can lead the application into failure. While doing some of the SME reviews to ecommerce projects, I also feel that developers might of done over architecture without knowing the strengths and weakness of catalog system. 

Catalog system contains following items.

  • Base Catalogs: Catalogs created in commerce server are base catalog.
  • Virtual Catalogs: Catalogs that are aggregated from one or more base or virtual catalogs.\
  • Catalog Sets: Grouping of one or more catalogs to target to the user is called catalog sets.
  • Categories: A category is a logical group of products or sub-categories.
  • Product/Product families: A product is a individual item in the catalog and if that product has multiple variants then it is called product family.

Microsoft has shared some of the architecture suggestions to be considered while developing CS2009 catalog system. Here are the few items which I collated from msdn site or documents published from Microsoft.

  • Supports up to 10,000 catalogs: 10,000 base catalogs, 10,000 virtual catalogs, or a combination of both base and virtual catalogs.
  • A catalog supports at least five million products. (CS has not been tested with 10,000 catalogs each containing five million products.)
  • A catalog can support up to 1,000 property definitions.
  • A virtual catalog can include content from up to 80 base catalogs.
  • A virtual catalog can include up to one million products.
  • A virtual catalog can support up to 10,000 price rules.
  • A category can support up to 10,000 products & can be nested up to five layers deep is recommended.
  • Multiple product catalogs can share a single SQL Server full-text catalog.
  • Add your own user-defined attributes at the catalog level and/or Category /Product level, as needed. Catalog attributes, such as currency, unit of weight measure, locale, and catalog name are available by default. You might want to add your own catalog attributes, for example, to store information about the vendor that supplies the products in a catalog.
  • Create separate catalogs (base or virtual) if search is to be restricted across a collection of products.
  • After you create a product, you cannot assign a different product definition to it. If you create a product based on product definition “mobile”, at a later date you cannot change the product to be based on product definition “Electronics”. You can, however, change product definition “mobile”, for example, by adding/removing a property.
  • You cannot change a data type of a property after the property is created.
  • You cannot assign a new product definition, or change a data type, after it is in use because other products or properties might be affected by the change.
  • If you need to search on a custom property, you should add an SQL index to that property.
  • Fewer catalogs are better, depending on hardware and your specific scenarios. This guideline is valid up to around 500,000 products per catalog.
  • A materialized catalog is a snapshot of a virtual catalog.
  • If the virtual catalog contains a number of base catalogs and/or the base catalogs contain a lot of products and categories then materializing the virtual catalog can result in better runtime performance.
  • If the content of the base catalogs changes frequently then materializing the virtual catalogs will mean frequent rebuilds which might offset the performance gain.
  • If the desired performance without materializing a virtual catalog meets your needs then you should not materialize the virtual catalog.
  • As a general rule of thumb : start without materializing a virtual catalog and determine based on whether  the desired performance is acceptable or not.
  • Do not add a product variant property to any properties that are being used as a Product Identifier property in a catalog. For example, if you use ModelNumber as the identifier for products in your catalog, do not create a product based on a product definition in which ModelNumber appears as a product variant property. If you do, you will not be able to identify variant information based on the product definition in that catalog.
  • Do not use the same value for product unique identifiers and variant unique identifiers. For example, you cannot assign stockkeeping unit (SKU) as the product unique identifier and the product variant unique identifier.
  • Before you import an XML catalog file, select the Validate XML file option on the Import Catalog dialog box to ensure that all data in your XML file is valid.
  • Start the full-text indexing service before you import a catalog. When a catalog is imported into catalog database, the full-text index is created at the same time.
  • If you experience poor performance when importing 100,000 to 1 million products into a base catalog, or publishing a virtual catalog of a similar size,
    run DBCC FREEPROCCACHE and sp_updatestats to improve the performance of APIs such as GetProduct and GetProductProperties:
  • If you create a .pup package that contains a materialized virtual catalog, after you unpack the virtual catalog, you must materialize the virtual catalog again.
  • Delete temporary tables from the MSCS_CatalogScratch database regularly. To improve run-time performance, the Product Catalog System uses the MSCS_CatalogScratch database to store temporary catalog tables.
  • Rebuild a virtual catalog in the following scenarios:
    • If a virtual catalog is materialized, when any data changes in the associated base catalogs.
    • If the virtual catalog is not materialized, whenever the data in the associated base catalogs changes (for example, pricing changes, and hierarchy and relationship changes).
    • You must rebuild a virtual catalog whenever you change include/exclude rules or pricing rules.
  • Do not rebuild a virtual catalog on a production server. Users will not be able to access the catalog. Only rebuild on staging server. After the catalogs are materialized, you can replicate them to you production servers.
  • Limit the number of rules in the virtual catalog. As a general rule you should rely on catalog and category level rules.
  • The list may be small and remaining good practices, you will learn in real time :) .

    Related Articles


    If you like this post, please click on our sponsor advertisement.


    Most of the medium and small ecommerce application support generic order process where the user has to add their products to shopping cart, manually checkout the basket, provide shipping, billing addresses and payment details. This is five page refresh process. The process will be applicable to the customer who wants to buy more products at once. Most of the customer will buy single product at a time and only few customers will perform bulk orders. To help single purchase buyers, we can implement Quick order.

    Quick Order (or single click ordering) is a very useful and powerful tool and can be built for ecommerce customers and customer service representatives (CSR), who works from back office.  By allowing customer to make their purchase quickly and in reduced steps will encourage them to come back and make another purchase in the future.  CSR can reduce the call time with the customer if s/he can order the product in reduced steps which improves efficiency.  Quick order application via the automated IVR system can provide 24×7 accesses for customer to purchase on phone. Quick order process will work well with the registered user and may/may not work with anonymous user.

    Registered User:
    View Product -> Click Quick Order button -> Order confirmation (if required) – > Order Display

    Non-register user:
    View Product -> Click Quick Order button -> Accept Billing & Shipping Address -> Accept Payment  -> Order confirmation (if required) – > Order Display For quick order process, we have to do some customizations in commerce server.

    1. Extend commerce server profile System to capture following details
      1. Default billing address 
      2. Default shipping address.
      3. Default Payment Method.  
      4. Default Credit/Debit card (if payment method is credit card)  
    2. In the registration form, try to accept following details from the customer
      1. Billing address
      2. Shipping address
      3. Payment details

    The application can use the information available in the user profile while performing quick checkout. So, when the user clicks on quick check button on product page, the following actions should take place.

    1. Input Values expected from UI – User Id, Catalog Name, and Product Id.
    2. Retrieve commerce server context of catalog, order and profile subsystems.
    3. Create new basket.
    4. Retrieve user profile properties like address Id, credit card Id, etc.
    5. Retrieve product details from Catalog subsystem and populate into basket.
    6. Retrieve address details from profile subsystem and populate into basket.
    7. Retrieve payment details (credit card) from Order subsystem and populate into basket.
    8. Execute pipelines in the following order and then convert basket into Purchase Order.  
      1. Basket  
      2. Total  
      3. Checkout  

    Click here to download Quick checkout code

    Related Articles


    If you like this post, please click on our sponsor advertisement.


    Ecommerce application needs product updates daily or frequently to show up to latest products information on their website. Sometimes the product information gets from company business users and sometimes the products information comes from vendors. For large scale ecommerce application, middleware servers like BizTalk are used for product import.

    This current tool (attached to this article) is command based tool and be perfect for small companies with limited data upload and can be used only one time (during application migration) or regularly. This tools takes catalog data using excel file and through commerce server API, it inserts data into database.

    The excel file template contains three sheets.

    1.    First sheet name  is “CategoryData” – This sheet contains categories details.
    2.    Second sheet name is “ProductData” – This sheet contains product details.
    3.     Third Sheet name is “VariantData” – This sheet contains variant data.

    Note: Kindly don’t change sheet names.

    When the tool is executed, it will ask for data excel file path. Once correct path is provided, it will ask whether to import category data, product data or variant data. Based on the option selected, the tool starts porting data into commerce server.

    The assumption is that the given catalog and product definition are created in commerce server.

    How to add a category?

    Category can be created under a Catalog only. So, before we create category, the application should get context of Catalog, then by using method CreateCategory(), category can be created.

    BaseCatalog baseCatalog = (BaseCatalog)context.GetCatalog((string)row["CatalogName"]);
    Category category = baseCatalog.CreateCategory((string)row["CategoryDefinition"], (string)row["CategoryName"]);
    category.DisplayName = (string)row["DisplayName"];
    category.Save();

    How to add a Product?

    Product can be created under catalog or category. So before we create product, the application should get catalog context. By using CreateProduct(), a product can be created. If the products need to be created under a category, then pass category information to CreateProduct() method.

    BaseCatalog baseCatalog = (BaseCatalog)context.GetCatalog((string)row["CatalogName"]);
    Product product = baseCatalog.CreateProduct((string)row["ProductDefinition"], (string)row["ProductID"], Convert.ToDecimal(row["ListPrice"]), (string)row["CategoryName"]);
    product.DisplayName = (string)row["DisplayName"];
    product["Image"] = row["Image"];
    product["Description"] = row["Description"];
    product.Save();

    How to add a variant?

    Products may or may not have a variant. For example, a shoe can have multiple variants like size, color, etc and some products don’t have variants like Books. To create a variant, the application should get product definition and to get product definition, the application needs to get catalog context. Once product definition obtained, we can add a new variant using AddVariant() method as shown below.

    BaseCatalog baseCatalog = (BaseCatalog)context.GetCatalog((string)row["CatalogName"]);
    ProductFamily productFamily = (ProductFamily)baseCatalog.GetProduct((string)row["ProductID"]);string variantID = (string)row["VariantID"];
    // Add the variant to the product family.
    productFamily.AddVariant(variantID);
    // Set the properties for the variant product. 
    productFamily.Variants[variantID]["VariantActualPrice"] = Convert.ToDecimal(row["VariantListPrice"]);
    productFamily.Variants[variantID]["Color"] = (string)row["VariantColor"];
    productFamily.Variants[variantID]["Size"] = (string)row["VariantSize"]
    productFamily.Variants[variantID]["VariantDisplayName"] = (string)row["VariantDisplayName"];
    productFamily.Save();

    If you have any questions/suggestions on his approach, feel free to send your comments. Click here to download Catalog Importer.

    Related Articles


    If you like this post, please click on our sponsor advertisement.


    Follow me on Google+
    Couldn't get data from google+
    Sign up for Newsletter