1.What is the difference between transfer learning and fine-tuning?

Transfer learning is the broader concept of reusing a model trained on one task for a new task. Fine-tuning is a specific step within transfer learning where you continue training the pre-trained model on new data, usually with a lower learning rate, to adjust its weights for the target task.

2. Can transfer learning be used with small datasets?

Yes, and that is one of its biggest strengths. Since the pre-trained model already carries general knowledge from large datasets, you need far fewer labeled examples to get good performance on a new task. Even a few hundred or thousand examples can be enough depending on how similar the tasks are.

3. What is a pre-trained model in transfer learning?

A pre-trained model is a neural network that was already trained on a large dataset for a specific task. Examples include ResNet trained on ImageNet for image classification, and BERT trained on large text corpora for language understanding. These models serve as the starting point in transfer learning.

4. What is negative transfer in machine learning?

Negative transfer happens when the knowledge from the source task hurts performance on the target task instead of helping. This usually occurs when the two tasks are too different. For example, a model trained on satellite images may not transfer well to detecting skin conditions from close-up photos.

5. Which pre-trained models are commonly used for NLP tasks?

BERT, GPT, RoBERTa, T5, and XLNet are among the most widely used pre-trained models for NLP. Each was trained on large text datasets and can be fine-tuned for tasks like text classification, summarization, translation, or question answering.

6. How many layers should I freeze during transfer learning?

depends on your dataset size and how similar it is to the original training data. If your dataset is small and similar to the source, freeze most layers and retrain only the last one or two. If your dataset is large and different, you can unfreeze more layers or fine-tune the entire model.

7. Is transfer learning only for deep learning?

Transfer learning is most commonly associated with deep learning because neural networks naturally build layered, transferable representations. But the concept also applies to other machine learning settings, such as adapting feature representations in traditional ML models, though the implementations differ.

8. What is domain adaptation in transfer learning?

Domain adaptation is a type of transfer learning where the task stays the same but the input data distribution changes between source and target. For example, a model trained on data from one hospital may need to be adapted to work well on data from a different hospital with different scanning equipment.

9. How does transfer learning reduce training costs?

By starting with a pre-trained model, you skip the most expensive phase of training where the model learns basic patterns from billions of examples. Fine-tuning a pre-trained model typically requires far less GPU time and fewer compute resources than training a comparable model from scratch.

10. What is feature extraction in the context of transfer learning?

Feature extraction is when you use the pre-trained model as a fixed feature extractor. You freeze all its layers and only train a new classifier on top of the features it produces. This is different from fine-tuning, where you actually update the weights of the base model during training.

11. When should you not use transfer learning?

Avoid transfer learning when your target task is fundamentally different from any available pre-trained model's source task. Also be cautious when you have a very large, unique dataset where training from scratch would give the model better task-specific representations without the constraints of the pre-trained model's architecture or biases.