We use the ArXiv Dataset with 11 labels that have the largest length variation across academic domains.<\/p>\n
\n
![\"Number](\"https:\/\/procycons.com\/wp-content\/uploads\/2025\/07\/dataset1.png\")
<\/p>\n<\/div>\n
Document Length Context<\/strong>: To better contextualize these word counts, we can convert them to number of pages, using the standard estimate of 500 words per page for double-spaced academic text (14,000 words \u2248 28 pages \u2248 short academic paper). By this measure:<\/p>\n\n- math.ST averages around 28 pages<\/li>\n
- math.GR and cs.DS are about 25\u201326 pages<\/li>\n
- cs.IT and math.AC average roughly 20\u201324 pages<\/li>\n
- while cs.CV and cs.NE average only 14\u201315 pages<\/li>\n<\/ul>\n
This significant variation demonstrates differences in writing styles, document depth, or research reporting norms across fields. Fields like mathematics and theoretical computer science tend to produce more comprehensive or technically dense documents, whereas applied areas like computer vision may favor more concise communication.<\/p>\n
<\/p>\n
\n
![\"Number](\"https:\/\/procycons.com\/wp-content\/uploads\/2025\/07\/dataset2.png\")
<\/p>\n
<\/p>\n<\/div>\n
3.2.2 Data Size and Training\/Test Division<\/h4>\n
Expected training time on standard hardware<\/strong>: 30 minutes to 8 hours, depending on method complexity.<\/p>\nMinimum Training Data Requirements<\/strong>:<\/p>\n\n- Simple methods: 50+ samples per class<\/li>\n
- Logistic Regression: 100+ samples per class<\/li>\n
- XGBoost: 1,000+ samples for optimal performance<\/li>\n
- BERT\/Transformer models: 2,000+ samples per class<\/li>\n<\/ul>\n
In all experiments, 30% of the data was reserved as the test set. To evaluate the robustness of the model, several variations of the dataset were used: the original class-distributed data, a balanced dataset based on the minimum class size (~2,505 samples), and additional balanced datasets with fixed sizes of 100, 140, and 1,000 samples per class.<\/p>\n
4. Results and Analysis<\/h2>\n
Our experiments reveal counter-intuitive results about the performance-efficiency trade-offs in long document classification.<\/p>\n
Why Traditional ML Outperforms Transformers<\/h3>\n
Our benchmark demonstrates that traditional machine learning approaches offer several advantages:<\/p>\n
\n- Computational Efficiency<\/strong>: Process entire documents without token limits<\/li>\n
![\"Keyword-Based](\"https:\/\/procycons.com\/wp-content\/uploads\/2025\/07\/Keyword-2.jpg\")
<\/p>\n<\/div>\n![\"TF-IDF-Based](\"https:\/\/procycons.com\/wp-content\/uploads\/2025\/07\/TFIDF-1.jpg\")
<\/p>\n![\"ML-Based](\"https:\/\/procycons.com\/wp-content\/uploads\/2025\/07\/ML-2.jpg\")
<\/p>\n![\"Transformer-Based](\"https:\/\/procycons.com\/wp-content\/uploads\/2025\/07\/Transformer-2.jpg\")
<\/p>\n![\"LLM-Based](\"https:\/\/procycons.com\/wp-content\/uploads\/2025\/07\/LLM-2.jpg\")
<\/p>\n<\/div>\n![\"LLM+RAG](\"https:\/\/procycons.com\/wp-content\/uploads\/2025\/07\/RAG-3-1.jpg\")
<\/p>\n<\/div>\n