{ "cells": [ { "cell_type": "markdown", "id": "b79cc765", "metadata": {}, "source": [ "# ★ 1-Star Battery Data\n", "\n", "This lesson demonstrates how to publish a battery dataset to the Zenodo Sandbox using their REST API. This corresponds to Star One in the Five-Star Battery Data recommendation.\n", "\n", "---\n", "\n", "## What does one-star mean? \n", "In the 5-Star Battery Data framework, 1-star data is:\n", "- Published to a public repository (e.g., Zenodo), and\n", "- Assigned a clear, permissive license for reuse (e.g., CC-BY 4.0)\n", "\n", "This notebook demonstrates how you can achieve your first star. \n", "\n", "---\n", "\n", "## Watch\n", "\n", "
\n", " \n", "
\n", "\n", "---\n", "\n", "## What is a public repository? \n", "A public repository is an open-access platform where research data can be deposited, described, and shared with others. It ensures that your dataset is openly accessible, citable, and preserved long-term by assigning a persistent identifier such as a DOI. Public repositories require key descriptive metadata—including title, abstract, keywords, licensing, and creator information—formatted in a machine-readable way to support indexing and reuse.\n", "Platforms like Zenodo support these features natively and allow metadata enrichment with identifiers such as ORCID (for authors) and ROR (for institutions), promoting clarity and credit attribution.\n", "\n", "---\n", "\n", "## Why is this important? \n", "Publishing your data in a public repository significantly increases its visibility, trustworthiness, and impact. A persistent identifier ensures others can reliably cite your dataset, while the repository guarantees long-term access and preservation. By including rich, standardized metadata, your dataset becomes easier to find, integrate, and reuse within research infrastructures, knowledge graphs, and semantic search tools—supporting the broader goals of open science and FAIR data.\n", "\n", "---\n", "\n", "## What we will do \n", "In this notebook, we will:\n", "1. **Load** your Zenodo Sandbox access token securely from a `.env` file\n", "2. **Define** dataset metadata including creator names, ORCID, affiliation, and license\n", "3. **Create** a deposition in the Zenodo Sandbox via the API\n", "4. **Upload** a structured `.csv` battery data file\n", "5. **Publish** the dataset and obtain a shareable DOI-like URL\n", "\n", "---\n" ] }, { "cell_type": "markdown", "id": "31d11da5", "metadata": {}, "source": [ "## What is the Zenodo Sandbox?\n", "\n", "The Zenodo Sandbox is a publicly available test server that mimics the real Zenodo publishing platform. It allows you to safely practice uploading datasets, registering metadata, and minting DOIs—without affecting the live site or making anything publicly visible.\n", "\n", "You can upload data in two ways:\n", "- Via the web interface – a user-friendly option ideal for one-time or manual uploads. \n", "- Via the API – a powerful method for automating uploads, especially useful when publishing many datasets or integrating Zenodo into your data processing pipeline. \n", "\n", "While the web interface is convenient, setting up a pipeline through the API is much more efficient and scalable for frequent or large-volume uploads. In this notebook, we will demonstrate how to use the API. \n", "\n", "**Zenodo Sandbox URL:** https://sandbox.zenodo.org\n", "\n", "---\n", "\n", "### What is an Access Token and Why Do I Need One?\n", "An access token is like a digital key that lets your code talk to Zenodo on your behalf. Instead of logging in with a username and password, you use this token to securely connect to Zenodo’s system—especially when using automated tools or scripts.\n", "\n", "It tells Zenodo who you are and what you're allowed to do, such as: \n", "- Uploading files \n", "- Editing metadata \n", "- Publishing or updating records\n", "\n", "This is essential when using the Zenodo Sandbox API to automate your data publishing workflow. Without an access token, Zenodo won’t know who’s making the request or whether they have permission.\n", "\n", "---\n", "\n", "### How to Create an Access Token\n", "\n", "Before using the API, you need a personal **access token** to authenticate your requests. Here’s how to create one:\n", "\n", "1. Go to [https://sandbox.zenodo.org](https://sandbox.zenodo.org) and sign in (you may need to register an account).\n", "2. Click your profile icon and choose **Applications**.\n", "\n", "![Alt text](img/zenodo_sandbox_applications.png)\n", "\n", "3. Click **New Token**.\n", "\n", "![Alt text](img/zenodo_sandbox_new_token.png)\n", "\n", "4. Give it a name like `\"five_star_data_test_token\"`.\n", "5. Enable the following scopes:\n", " - `deposit:write` (upload new data)\n", " - `deposit:actions` (publish data)\n", " - `user:email` (optional, to identify yourself)\n", "6. Click **Create** and copy the generated token.\n", "\n", "![Alt text](img/zenodo_sandbox_create_token.png)\n", "\n", "---\n", "\n", "### Load Access Token\n", "\n", "To keep your token secure, you should store it in a `.env` file rather than hardcoding it into this notebook. Open the ```.env``` file that accompanies this notebook and paste your token into the following field:\n", "\n", "```env\n", "ZENODO_SANDBOX_TOKEN=paste_your_sandbox_token_here\n", "```\n", "\n", "---\n" ] }, { "cell_type": "code", "execution_count": 9, "id": "45a2857f", "metadata": {}, "outputs": [], "source": [ "# ====================\n", "# 🛠 LOAD DEPENDENCIES\n", "# ====================\n", "import os\n", "import sys\n", "import requests\n", "from dotenv import load_dotenv\n", "from IPython.display import display, Markdown\n", "import re" ] }, { "cell_type": "code", "execution_count": 10, "id": "2bda791b", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "✅ Access token loaded\n" ] } ], "source": [ "\n", "# ====================\n", "# 🛠 LOAD ACCESS TOKEN\n", "# ====================\n", "load_dotenv(override=True)\n", "ACCESS_TOKEN = os.getenv(\"ZENODO_SANDBOX_TOKEN\")\n", "DEFAULT_PLACEHOLDER = \"paste_your_sandbox_token_here\"\n", "\n", "# ============================\n", "# ✅ VALIDATE ACCESS TOKEN\n", "# ============================\n", "if not ACCESS_TOKEN or ACCESS_TOKEN == DEFAULT_PLACEHOLDER:\n", " print(\"\\n❌ Access token is missing or still set to the default placeholder.\")\n", " print(\"👉 Please open your `.env` file and replace the placeholder with your actual Zenodo Sandbox token:\")\n", " print(\" ZENODO_SANDBOX_TOKEN=your_actual_token_here\\n\")\n", " sys.exit(1)\n", "\n", "print(f\"✅ Access token loaded\")\n", "\n", "# ====================\n", "# 🔗 ZENODO SANDBOX API\n", "# ====================\n", "ZENODO_SANDBOX_URL = \"https://sandbox.zenodo.org/api/deposit/depositions\"\n", "HEADERS = {\n", " \"Content-Type\": \"application/json\",\n", " \"Authorization\": f\"Bearer {ACCESS_TOKEN}\"\n", "}\n" ] }, { "cell_type": "markdown", "id": "0e0414c5", "metadata": {}, "source": [ "## Define metadata\n", "\n", "Before uploading a file to Zenodo, you must define the metadata that describes your dataset. This metadata helps make the dataset **searchable, citable, and understandable**. Zenodo expects metadata in a structured format, aligned with community standards like schema.org.\n", "\n", "In this example, we define the following key fields:\n", "\n", "- `title`: A descriptive name for the dataset \n", "- `upload_type`: Specifies the content type (e.g., `\"dataset\"`, `\"software\"`, `\"publication\"`) \n", "- `description`: A brief abstract or summary of what the dataset contains \n", "- `creators`: A list of contributors including:\n", " - `name`: Full name \n", " - `affiliation`: Institutional affiliation \n", " - `orcid`: ORCID identifier (machine-readable researcher ID) \n", " - `affiliation_ror`: ROR identifier for the institution (structured organizational ID) \n", "- `keywords`: Tags that make the record more discoverable \n", "- `access_right`: `\"open\"`, `\"embargoed\"`, `\"restricted\"`, or `\"closed\"` depending on data availability \n", "- `license`: Specifies how the data can be reused (e.g., `\"CC-BY-4.0\"` for Creative Commons Attribution)\n", "\n", "\n", "\n", "> **✏️ Customize your metadata** \n", "> \n", "> Before running the notebook, make sure to update the metadata fields with your own information. Replace the following placeholders:\n", "> \n", "> - `\"Last Name, First Name\"` → Your full name \n", "> - `\"Organization Name\"` → Your current institution or affiliation \n", "> - `\"https://orcid.org/YOUR-ORCID-NUMBER\"` → Your personal [ORCID](https://orcid.org) \n", "> - `\"https://ror.org/YOUR-AFFILIATION-ROR-ID\"` → Your organization's [ROR ID](https://ror.org) \n", "> \n", "> If you don't have an ORCID or ROR ID, you can temporarily remove those fields, but we recommend including them for better interoperability.\n", ">\n", "> For a full description of all metadata fields and API endpoints, see the [Zenodo REST API documentation](https://developers.zenodo.org). " ] }, { "cell_type": "code", "execution_count": 11, "id": "9ef399b6", "metadata": {}, "outputs": [], "source": [ "# ====================\n", "# 📝 METADATA\n", "# ====================\n", "metadata = {\n", " \"metadata\": {\n", " \"title\": \"Example Battery Dataset\",\n", " \"upload_type\": \"dataset\",\n", " \"description\": \"A simple CSV file representing battery time series data.\",\n", " \"creators\": [\n", " {\n", " \"name\": \"Clark, Simon\",\n", " \"affiliation\": \"SINTEEF\",\n", " \"orcid\": \"https://orcid.org/0000-0002-8758-6109\",\n", " \"affiliation_ror\": \"https://ror.org/01f677e56\"\n", " }\n", " ],\n", " \"keywords\": [\"battery\", \"time series\", \"example\"],\n", " \"access_right\": \"open\",\n", " \"license\": \"CC-BY-4.0\"\n", " }\n", "}" ] }, { "cell_type": "code", "execution_count": 12, "id": "fe9af430", "metadata": {}, "outputs": [], "source": [ "# ============================\n", "# VALIDATE ORCID AND ROR\n", "# ============================\n", "\n", "# This block is checking to validate that you have provided a value for the ORCID and RORID in the metadata snippet above.\n", "\n", "# Regular expression patterns\n", "orcid_pattern = r\"^https:\\/\\/orcid\\.org\\/\\d{4}-\\d{4}-\\d{4}-\\d{4}$\"\n", "ror_pattern = r\"^https:\\/\\/ror\\.org\\/[0-9a-z]{9}$\"\n", "\n", "creator = metadata[\"metadata\"][\"creators\"][0]\n", "orcid = creator.get(\"orcid\", \"\")\n", "rorid = creator.get(\"affiliation_ror\", \"\")\n", "\n", "# Check if ORCID and ROR ID are valid\n", "valid_orcid = re.match(orcid_pattern, orcid)\n", "valid_rorid = re.match(ror_pattern, rorid)\n", "\n", "if not valid_orcid or not valid_rorid:\n", " display(Markdown(\"\"\"\n", "**❌ ORCID or ROR ID is missing or invalid.** \n", "👉 Please update the metadata with your actual, properly formatted identifiers.\n", "\n", "- ORCID should look like: `https://orcid.org/0000-0002-1825-0097` \n", "- ROR ID should look like: `https://ror.org/05gq02987`\n", "\"\"\"))\n", " sys.exit(1)" ] }, { "cell_type": "markdown", "id": "490f27e7", "metadata": {}, "source": [ "## Create a Draft Record\n", "\n", "Once the metadata is defined, the next step is to create **a private draft record** in Zenodo (also called a **\"deposition\"**) that holds your data and metadata before publication. We use a `POST` request to the Zenodo API, sending the metadata as JSON along with the authentication headers. If the request is successful (`status_code == 201`), Zenodo returns a JSON response containing the deposition ID and other metadata.\n", "\n", "```python\n", "response = requests.post(ZENODO_SANDBOX_URL, json=metadata, headers=HEADERS)\n", "```\n", "\n", "We extract the `deposition_id` from the response. This ID is required to:\n", "- Upload files to the correct draft record\n", "- Refer to the deposition in later actions (like publishing or deleting)\n", "\n", "If the deposition is not created successfully, the script prints the error message and stops.\n" ] }, { "cell_type": "code", "execution_count": 13, "id": "2b70c6b7", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "✅ Created deposition: 409313\n" ] } ], "source": [ "# ====================\n", "# 📤 CREATE DEPOSITION\n", "# ====================\n", "response = requests.post(ZENODO_SANDBOX_URL, json=metadata, headers=HEADERS)\n", "if response.status_code == 201:\n", " deposition = response.json()\n", " deposition_id = deposition[\"id\"]\n", " print(f\"✅ Created deposition: {deposition_id}\")\n", "else:\n", " print(\"❌ Failed to create deposition:\", response.text)\n", " exit(1)" ] }, { "cell_type": "markdown", "id": "664edeb5", "metadata": {}, "source": [ "## Upload a file\n", "\n", "After creating a deposition, the next step is to upload the dataset file. In this example, we upload a structured `.csv` file that follows the Battery Data Format (BDF) standard. We start by defining the path to the file and extracting the filename. Then we open the file in binary mode and use a `POST` request to send it to Zenodo. The upload endpoint is based on the deposition ID obtained earlier. If the upload is successful (`status_code == 201`), a confirmation message is printed. If not, the error is displayed and the script exits.\n", "\n", "> ℹ️ **Note:** \n", "> You can upload multiple files to the same deposition by repeating this process. All files must be uploaded **before** publishing the record.\n" ] }, { "cell_type": "code", "execution_count": 14, "id": "88a91b01", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "✅ File 'structured_battery_data.bdf.csv' uploaded successfully.\n" ] } ], "source": [ "# ====================\n", "# 📎 UPLOAD FILE\n", "# ====================\n", "file_path = \"structured_battery_data.bdf.csv\"\n", "filename = os.path.basename(file_path)\n", "\n", "with open(file_path, \"rb\") as file:\n", " files = {\"file\": (filename, file)}\n", " upload_url = f\"{ZENODO_SANDBOX_URL}/{deposition_id}/files\"\n", " r = requests.post(upload_url, files=files, headers={\"Authorization\": f\"Bearer {ACCESS_TOKEN}\"})\n", " if r.status_code == 201:\n", " print(f\"✅ File '{filename}' uploaded successfully.\")\n", " else:\n", " print(\"❌ File upload failed:\", r.text)\n", " exit(1)" ] }, { "cell_type": "markdown", "id": "c7709f05", "metadata": {}, "source": [ "## Publish the record\n", "\n", "Once the metadata and file upload steps are complete, the final step in the Zenodo workflow is to **publish** the deposition. This action finalizes the dataset and makes it publicly accessible in the Zenodo Sandbox. Publishing mimics what you would do in a real research scenario, where a DOI is minted and the record becomes part of a public repository.\n", "\n", "> **🚨 Note:** \n", "> \n", "> Records that are published in the sandbox **cannot be deleted**! We strongly recommend using a dry run for test uploads.\n", "\n", "### Use dry run mode to avoid polluting the sandbox\n", "\n", "To give students or developers the **full experience of creating and uploading metadata and files** without leaving clutter behind, we include a `dry_run` option in the script. When `dry_run = True`, the script will:\n", "\n", "- Go through all the steps: load token, define metadata, create a deposition, and upload a file.\n", "- **Stop before publishing**, and instead **delete the draft deposition**.\n", "- Print a confirmation that the deposition was deleted.\n", "\n", "```python\n", "dry_run = True\n", "```\n", "If you would like to publish a real record to the sandbox to get the full effect, then set: \n", "\n", "```python\n", "dry_run = False\n", "```\n", "\n", "You should only do this once, to avoid creating duplicate records.\n" ] }, { "cell_type": "code", "execution_count": 15, "id": "3de575f8", "metadata": {}, "outputs": [], "source": [ "# ====================\n", "# SET DRY RUN VARIABLE\n", "# ====================\n", "\n", "dry_run = False" ] }, { "cell_type": "code", "execution_count": 16, "id": "be73aaa5", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "🎉 Dataset published: https://sandbox.zenodo.org/record/409313\n" ] } ], "source": [ "\n", "# ====================\n", "# ✅ PUBLISH OR DELETE\n", "# ====================\n", "\n", "if dry_run:\n", " print(\"🧹 Dry run enabled. Deleting test deposition...\")\n", " r = requests.delete(f\"{ZENODO_SANDBOX_URL}/{deposition_id}\", headers=HEADERS)\n", " if r.status_code == 204:\n", " print(\"🗑️ Test deposition deleted successfully.\")\n", " else:\n", " print(\"⚠️ Failed to delete test deposition:\", r.text)\n", "else:\n", " publish_url = f\"{ZENODO_SANDBOX_URL}/{deposition_id}/actions/publish\"\n", " r = requests.post(publish_url, headers=HEADERS)\n", "\n", " if r.status_code == 202:\n", " print(f\"🎉 Dataset published: https://sandbox.zenodo.org/record/{deposition_id}\")\n", " else:\n", " print(\"❌ Failed to publish dataset:\", r.text)" ] }, { "cell_type": "markdown", "id": "3afbfb80", "metadata": {}, "source": [ "## Summary\n", "\n", "In this notebook, you learned how to publish a structured battery dataset to the Zenodo Sandbox using their REST API. This hands-on workflow walks through every step needed to achieve **1-star battery data** in the Five-Star Battery Data framework:\n", "\n", "| Step | What You Did |\n", "|--------------------------|--------------------------------------------------------------|\n", "| Create a token | Generated a personal access token from the Zenodo Sandbox |\n", "| Define metadata | Structured your dataset description using standard fields |\n", "| Create a deposition | Created a new draft record to hold your files and metadata |\n", "| Upload your data | Uploaded a `.csv` file representing structured battery data |\n", "| Ran a dry run | Practiced safely by deleting test records before publishing |\n", "| (Optional) Publish | Finalized the record and generated a DOI-like URL |\n", "\n", "By following this process, you've made your dataset:\n", "- Publicly accessible\n", "- Citable with a stable identifier\n", "- Reusable under a clear license\n", "- Discoverable through structured metadata \n", "\n", "This notebook gives you a complete and reusable pattern for publishing scientific datasets in a FAIR and standards-aligned way.\n" ] }, { "cell_type": "markdown", "id": "dd0ce978", "metadata": {}, "source": [ "---\n", "\n", "\"EU\n", "\n", "**This work has received funding from the European Union under the Horizon Europe programme.** \n", "Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Commission. Neither the European Union nor the granting authority can be held responsible for them." ] } ], "metadata": { "kernelspec": { "display_name": ".venv", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.10" } }, "nbformat": 4, "nbformat_minor": 5 }