Daily Literature Search Skill

Automated literature search system for academic researchers. Performs scheduled searches across multiple databases (PubMed, OpenAlex, Semantic Scholar), automatically deduplicates results, downloads open-access papers, and generates daily reports.

🎯 Use Cases

• Daily literature monitoring for specific research topics
• Automated paper collection for literature reviews
• Stay updated on latest publications in your field
• Build personal paper library with automatic categorization

📦 Components

1. Core Search Script (daily_literature_search.py)

Main execution script with the following features:

• Multi-source search: PubMed, OpenAlex, Semantic Scholar
• Automatic deduplication: By DOI (within batch + against local library)
• OA detection: Uses Unpaywall API to identify open-access papers
• Auto-download: Downloads OA papers from PubMed Central or publisher sites
• Smart categorization: Classifies papers by topic (configurable keywords)
• Daily reports: Generates Markdown reports with search statistics

2. Upload Analyzer (analyze_uploaded.py)

Analyzes and categorizes manually uploaded papers:

• Filename-based classification: Uses keyword matching
• DOI extraction: From filenames and metadata
• Batch processing: Handles multiple files at once
• Report generation: Creates categorization summary

⚙️ Configuration

Directory Structure

papers/
├── B-ALL/raw/          # Category 1 (e.g., B-ALL research)
├── MM/raw/             # Category 2 (e.g., Multiple Myeloma)
├── OTHER/raw/          # Other papers
├── daily_search_logs/  # Search logs and reports
└── upload_temp/        # Temporary upload directory

Search Keywords (Customizable)

Edit SEARCH_KEYWORDS in daily_literature_search.py:

SEARCH_KEYWORDS = [
    '"inotuzumab ozogamicin"',
    '"Elranatamab"',
    '"Teclistamab"',
    '"Talquetamab"',
    '"Blinatumomab"',
    '("CAR-T" AND "B-ALL")',
]

Classification Keywords

Edit B_ALL_KEYWORDS and MM_KEYWORDS in analyze_uploaded.py to match your research domains.

🚀 Usage

Manual Execution

# Run daily search
python3 papers/daily_literature_search.py

# Analyze uploaded papers
python3 papers/analyze_uploaded.py

Scheduled Execution (Cron)

Add to crontab for automatic daily searches:

# Daily search at 6:30 AM
30 6 * * * /usr/bin/python3 /path/to/papers/daily_literature_search.py >> /path/to/papers/daily_search_logs/cron.log 2>&1

Configuration Options

📊 Output

Daily Report Example

# 📚 每日文献检索报告
**检索日期：** 2026-03-18

## 📊 检索汇总
| 分类 | 检索到 | 成功下载 | 付费墙 |
|------|--------|---------|--------|
| B-ALL | 28 | 0 | 28 |
| MM | 24 | 0 | 24 |
| 总计 | 53 | 0 | 53 |

## 🔀 去重统计
- 原始检索结果：130 篇
- 去重后文献：110 篇
- 批次内重复：2 篇
- 库中已有：18 篇

File Organization

• Reports: papers/daily_search_logs/daily_report_YYYY-MM-DD.md
• Logs: papers/daily_search_logs/daily_search_YYYY-MM-DD.log
• Papers: papers/{CATEGORY}/raw/{DOI}.pdf

🔧 Advanced Features

1. Library Deduplication

Automatically checks new results against existing library:

• Scans all category directories for existing DOIs
• Extracts DOIs from filenames and historical logs
• Skips papers already in library
• Reports duplicate statistics

2. Open Access Detection

Uses Unpaywall API to identify OA papers:

is_oa, oa_url = check_open_access(doi)
if is_oa:
    download_paper(oa_url, save_path)

3. PubMed Central Integration

Automatically tries PMC for biomedical papers:

if pmid and str(pmid).isdigit():
    download_from_pubmed(pmid, save_path)

🛠️ Customization Guide

Change Research Topics

1. Edit SEARCH_KEYWORDS in daily_literature_search.py
2. Update category names and keywords
3. Modify directory structure if needed

Add New Categories

1. Create new directory: papers/NEW_CATEGORY/raw/
2. Add classification keywords in classify_paper() function
3. Update report generation to include new category

Integrate with Notification Systems

Add email/Slack/Discord notifications after search completion:

# At end of main()
send_notification(f"Daily search complete: {results['total']} papers found")

📋 Requirements

Python Dependencies

pip install requests
# Most other modules are standard library

API Access (Optional but Recommended)

• Semantic Scholar API Key: Higher rate limits
• OpenAlex API Key: Polite pool access
• Unpaywall: Free, no key needed (email required)

Set environment variables:

export SEMANTIC_SCHOLAR_API_KEY="your-key"
export OPENALEX_API_KEY="your-key"
export USER_EMAIL="[email protected]"

⚠️ Important Notes

1. Rate Limits: Respect API rate limits, especially without API keys
2. Storage: Monitor disk space for downloaded PDFs
3. Copyright: Only download open-access or legally available papers
4. Email: Set USER_EMAIL for polite API access

🔄 Version History

• 1.0.0 (2026-03-18): Initial release
  • Multi-source search (PubMed, OpenAlex, Semantic Scholar)
  • Automatic deduplication (batch + library)
  • OA detection and download
  • Smart categorization
  • Daily reports with statistics

🤝 Contributing

To contribute improvements:

1. Fork the skill repository
2. Test changes with your own literature search
3. Submit pull request with description of improvements

📄 License

This skill is provided as-is for academic research purposes. Users are responsible for compliance with publisher terms and copyright laws.