The risks of foundation models: Outsourced innovation

Dataset alignment can also be a challenge for those using foundation models. Pre-training on a large general-purpose dataset is no guarantee that the network will be able to perform a new task on proprietary data. The network may be so lacking in context or biased based on its pre-training, that even fine-tuning may not readily resolve the issue. 

Any startup leveraging foundation models in its machine learning efforts should pay close attention to these types of issues.