Custom Products under Product Maintenance are a legacy field that are no longer used in anything besides gift cards. The option to check this is now hidden since it serves no use.
Custom Products were a way to add in specific custom items to an order as an administrator without clogging up your product catalog. The product catalog actually will exclude these items from being in stock, sort of like marking an item inactive.
The original purpose of it, in the first retail store in Coreware, was for the bookstore. The client would get requests for products that he could special order, but didn't want to put in his catalog. Using a custom product, you could set the description, the price and put it in the customer's shopping cart. The bookstore industry is a bit different. There are a lot more products and distributors don't dropship... So your catalog has FAR less in it. Yet, you could1 special order nearly anything.
With Custom Products, the price could be set per order item rather than per product. A custom product essentially can have any price. We also used these in the order history import when we couldn’t match a product with anything else. Additionally, the order item description could be whatever.
It was never implemented (because it was never needed) in the new ecommerce tools.
This would not be difficult to implement if there is ever a need for it.
Use in Gift Cards
The only product that should have this box checked is gift cards (due to the variable pricing), and this will be done automatically via Orders -> Setup.
If you'd like to learn more about gift cards in coreFORCE you can click Here.