Press Release
Neurotechnology Explores Continual Learning to Expand Large Language Model Capacity
Neurotechnology's new research demonstrates how the application of continual learning advances LLMs' linguistic capabilities.
Vilnius, Lithuania – June 10, 2025 – Neurotechnology, a provider of deep-learning-based solutions and high-precision biometric identification technologies, today presented research results that allow artificial intelligence (AI) models to gain new knowledge while minimizing the loss of previously learned information. This research is based on continual learning, a method that enables AI models to be trained more efficiently.
Traditionally, incorporating new information into AI systems relying on neural networks has posed a significant challenge: training on new tasks often leads to a phenomenon known as catastrophic forgetting, where the model loses knowledge of previously learned tasks. To avoid this, models typically must be retrained from scratch - a time-consuming and usually large, resource-demanding process.
Neurotechnology conducted a large-scale continual learning experiment in which researchers integrated additional linguistic fluency in the Lithuanian language into the full parameter set of the Gemma2 large language model. The experiment simultaneously preserved the model's existing domain knowledge by inhibiting updates of its parameters that are related to the Massive Multitask Language Understanding (MMLU) task.
The results of this experiment reveal that it allowed balancing of the retention of the existing models' capabilities with the acquisition of new ones, in terms of both fluency and domain knowledge. Post-training evaluations demonstrated improved generation quality in Lithuanian while maintaining strong performance across the language understanding benchmarks.
"Continual learning methods have the potential to drive more adaptive and sustainable AI development," said Artūras Nakvosas, Research and Development Team Lead at Neurotechnology. "Our findings demonstrate that it's possible to mitigate catastrophic forgetting and optimize learning efficiency across multilingual and task-specific domains."
The benefits of continual learning are wide-ranging. With this approach, large language model developers can potentially expand models' performance, support additional customer languages or adapt to new domains while preserving existing capacities.
The full technical report detailing Neurotechnology's findings is available on arXiv at https://www.arxiv.org/abs/2505.05946, and the code used in the research is available on GitHub at https://github.com/Neurotechnology/LLM_EWC.
About Neurotechnology
Neurotechnology is a developer of high-precision algorithms and software based on deep neural networks and other AI-related technologies. The company was launched in 1990 in Vilnius, Lithuania, with the key idea of leveraging neural network capabilities for various applications, such as biometric person identification, natural language processing (NLP), computer vision, and artificial intelligence. The company focuses heavily on continuous research and development to advance the capabilities of artificial intelligence technologies. With a team of scientists and engineers, Neurotechnology consistently explores new methods in deep learning, computer vision, natural language processing, and neural network optimization. This ongoing innovation allows the company to deliver cutting-edge solutions across a wide range of applications.
Media Contact
Jennifer Allen Newton
Bluehouse Consulting Group, Inc.
+1-503-805-7540
jennifer (at) bluehousecg (dot) com
