Posts Tagged ‘developing catalog system’

For past few weeks, I am seeing few architecture questions in MSDN communities on how to mold catalog system to address client requirements. Catalog system is heart of the ecommerce solution and any architecture flaw can lead the application into failure. While doing some of the SME reviews to ecommerce projects, I also feel that developers might of done over architecture without knowing the strengths and weakness of catalog system. 

Catalog system contains following items.

  • Base Catalogs: Catalogs created in commerce server are base catalog.
  • Virtual Catalogs: Catalogs that are aggregated from one or more base or virtual catalogs.\
  • Catalog Sets: Grouping of one or more catalogs to target to the user is called catalog sets.
  • Categories: A category is a logical group of products or sub-categories.
  • Product/Product families: A product is a individual item in the catalog and if that product has multiple variants then it is called product family.

Microsoft has shared some of the architecture suggestions to be considered while developing CS2009 catalog system. Here are the few items which I collated from msdn site or documents published from Microsoft.

  • Supports up to 10,000 catalogs: 10,000 base catalogs, 10,000 virtual catalogs, or a combination of both base and virtual catalogs.
  • A catalog supports at least five million products. (CS has not been tested with 10,000 catalogs each containing five million products.)
  • A catalog can support up to 1,000 property definitions.
  • A virtual catalog can include content from up to 80 base catalogs.
  • A virtual catalog can include up to one million products.
  • A virtual catalog can support up to 10,000 price rules.
  • A category can support up to 10,000 products & can be nested up to five layers deep is recommended.
  • Multiple product catalogs can share a single SQL Server full-text catalog.
  • Add your own user-defined attributes at the catalog level and/or Category /Product level, as needed. Catalog attributes, such as currency, unit of weight measure, locale, and catalog name are available by default. You might want to add your own catalog attributes, for example, to store information about the vendor that supplies the products in a catalog.
  • Create separate catalogs (base or virtual) if search is to be restricted across a collection of products.
  • After you create a product, you cannot assign a different product definition to it. If you create a product based on product definition “mobile”, at a later date you cannot change the product to be based on product definition “Electronics”. You can, however, change product definition “mobile”, for example, by adding/removing a property.
  • You cannot change a data type of a property after the property is created.
  • You cannot assign a new product definition, or change a data type, after it is in use because other products or properties might be affected by the change.
  • If you need to search on a custom property, you should add an SQL index to that property.
  • Fewer catalogs are better, depending on hardware and your specific scenarios. This guideline is valid up to around 500,000 products per catalog.
  • A materialized catalog is a snapshot of a virtual catalog.
  • If the virtual catalog contains a number of base catalogs and/or the base catalogs contain a lot of products and categories then materializing the virtual catalog can result in better runtime performance.
  • If the content of the base catalogs changes frequently then materializing the virtual catalogs will mean frequent rebuilds which might offset the performance gain.
  • If the desired performance without materializing a virtual catalog meets your needs then you should not materialize the virtual catalog.
  • As a general rule of thumb : start without materializing a virtual catalog and determine based on whether  the desired performance is acceptable or not.
  • Do not add a product variant property to any properties that are being used as a Product Identifier property in a catalog. For example, if you use ModelNumber as the identifier for products in your catalog, do not create a product based on a product definition in which ModelNumber appears as a product variant property. If you do, you will not be able to identify variant information based on the product definition in that catalog.
  • Do not use the same value for product unique identifiers and variant unique identifiers. For example, you cannot assign stockkeeping unit (SKU) as the product unique identifier and the product variant unique identifier.
  • Before you import an XML catalog file, select the Validate XML file option on the Import Catalog dialog box to ensure that all data in your XML file is valid.
  • Start the full-text indexing service before you import a catalog. When a catalog is imported into catalog database, the full-text index is created at the same time.
  • If you experience poor performance when importing 100,000 to 1 million products into a base catalog, or publishing a virtual catalog of a similar size,
    run DBCC FREEPROCCACHE and sp_updatestats to improve the performance of APIs such as GetProduct and GetProductProperties:
  • If you create a .pup package that contains a materialized virtual catalog, after you unpack the virtual catalog, you must materialize the virtual catalog again.
  • Delete temporary tables from the MSCS_CatalogScratch database regularly. To improve run-time performance, the Product Catalog System uses the MSCS_CatalogScratch database to store temporary catalog tables.
  • Rebuild a virtual catalog in the following scenarios:
    • If a virtual catalog is materialized, when any data changes in the associated base catalogs.
    • If the virtual catalog is not materialized, whenever the data in the associated base catalogs changes (for example, pricing changes, and hierarchy and relationship changes).
    • You must rebuild a virtual catalog whenever you change include/exclude rules or pricing rules.
  • Do not rebuild a virtual catalog on a production server. Users will not be able to access the catalog. Only rebuild on staging server. After the catalogs are materialized, you can replicate them to you production servers.
  • Limit the number of rules in the virtual catalog. As a general rule you should rely on catalog and category level rules.
  • The list may be small and remaining good practices, you will learn in real time :) .

    Related Articles


    If you like this post, please click on our sponsor advertisement.


    Follow me on Google+
    Couldn't get data from google+
    Sign up for Newsletter