How Model Distillation May Drive DeepSeek and IP Concerns

DeepSeek has upset the world of AI, providing comparable results of leading AI models at a fraction of the cost. DeepSeek developers credit new reinforced learning techniques for the achievement. However, many believe that model distillation is at the core of DeepSeek’s strategy. Model distillation is when a “student” AI model is trained on the outputs of a “teacher” model. Distillation can be done by “parent” developers to create smaller more efficient models. It can also be done by third parties acting without authorization creating cheap alternatives. Notably many AI outputs are not protected by copyright, as I discussed in yesterday’s blog. This makes distillation a difficult strategy to counter. However, a recent memo from Fenwick suggests that patent law may hold some protections:

“With the right planning and strategy, a patentee may be able to secure protection not only for the teacher model but also against unauthorized student models derived through distillation. Although an inventor may not contemplate unauthorized distilled models as the main embodiment of the invention, they can sometimes still claim those distilled models as part of the patent along with or in addition to the teacher model”

While patents remain unproven as a tool to protect against distillation, the memo makes a strong case for their application. Additionally, AI developers can take a series of practical steps to avoid unauthorized distillation. These include limiting access to AI model APIs, as well as express prohibitions against distillation in terms of service agreements. Ultimately distillation may prove a difficult tactic to counter, as unauthorized distillation is not always obvious and may occur beyond the jurisdiction of IP enforcement.