Please rotate your device to landscape to enjoy better experience.

Exploring SPIN

Sparsifying and Integrating Internal Neurons in Large Language Models for Text Classification

Difan Jiao, Yilun Liu, Zhenwei Tang, Daniel Matter, Jürgen Pfeffer, Ashton Anderson

At the beginning...

For each given sentence, we retrieve its neuron activation patterns of FFN neurons at each layer of the transformer model.

Pooling Operation

A max pooling is then performed across all tokens, yielding each neuron's maximum activation record for the sentence.

Layer-Wise Linear Probes

Linear probing is applied layer-wise, identifying salient neurons for the targeted task among each layer of the model.

The magnitudes of learned L1-regularized logistic regression weights indicate the relative contribution to the prediction, i.e., saliency, of each neuron.

Salient Neuron Selection

The internal neurons are then sparsified based on their saliency. Guided by a sparsification threshold η, we calculate the cumulative contribution to have the most significant neurons.

Salient Neuron Integration

For the sparsified internal neurons from each respective layer, we proceed to integrate them as cross-layer multi-grained representations into the classification head.

Classification Head

For each layer, the integrated neurons from that layer and below are concatenated and feed into the classification head.