Skip to content

Commit 62ebf7f

Browse files
VaghinakDevVaghinakDev
authored
Merge pull request #754 from superannotateai/FRIDAY_3537
Friday 3537
2 parents 37462a2 + 17538ff commit 62ebf7f

File tree

4 files changed

+266
-11
lines changed

4 files changed

+266
-11
lines changed

‎docs/source/userguide/SDK_Functions_sheet.csv‎

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
Topic,Function,MM(GenAI/LLM),CSV,JSON,JSONL,INTEGRATION
2+
Projects,create_project,No,Not Relevant,Not Relevant,Not Relevant,Not Relevant
3+
,search_projects,Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
4+
,clone_project,No,Not Relevant,Not Relevant,Not Relevant,Not Relevant
5+
,rename_project,Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
6+
,delete_project,Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
7+
,get_project_by_id,Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
8+
,set_project_status,Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
9+
,get_project_metadata,Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
10+
,upload_images_to_project,Not Relevant,Not Relevant,Not Relevant,Not Relevant,Not Relevant
11+
,attach_items_from_integrated_storage,Yes,Not Relevant,Not Relevant,Not Relevant,"AWS, GCP, Azure"
12+
,upload_image_to_project,Not Relevant,Not Relevant,Not Relevant,Not Relevant,Not Relevant
13+
,upload_images_from_folder_to_project,Not Relevant,Not Relevant,Not Relevant,Not Relevant,AWS
14+
,upload_video_to_project,Not Relevant,Not Relevant,Not Relevant,Not Relevant,Not Relevant
15+
,upload_videos_from_folder_to_project,Not Relevant,Not Relevant,Not Relevant,Not Relevant,Not Relevant
16+
,set_project_custom_field,Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
17+
,add_contributors_to_project,Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
18+
,get_project_settings,Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
19+
,set_project_default_image_quality_in_editor,Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
20+
,set_project_steps,Not Relevant,Not Relevant,Not Relevant,Not Relevant,Not Relevant
21+
,get_project_steps,Not Relevant,Not Relevant,Not Relevant,Not Relevant,Not Relevant
22+
,get_component_config,Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
23+
Folders,search_folders(),Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
24+
,set_folder_status(),Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
25+
,assign_folder(),Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
26+
,unassign_folder(),Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
27+
,get_folder_by_id(),Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
28+
,get_folder_metadata(),Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
29+
,create_folder(),Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
30+
,delete_folders(,Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
31+
Items,query(),Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
32+
,get_item_by_id(),Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
33+
,list_items(),Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
34+
,search_items(),Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
35+
,attach_items(),Not Relevant,Not Relevant,No,No,Not Relevant
36+
,item_context(),Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
37+
,copy_items(),Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
38+
,move_items(),Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
39+
,delete_items(),Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
40+
,assign_items(),Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
41+
,unassign_items(),Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
42+
,get_item_metadata(),Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
43+
,set_approval_statuses(),Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
44+
Annotations,upload_annotations(),Yes,Yes,Yes,Yes,Not Relevant
45+
,get_annotations(),Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
46+
,download_annotations(),Yes,No,Yes,No,Not Relevant
47+
,get_annotations_per_frame(),Not Relevant,Not Relevant,Not Relevant,Not Relevant,Not Relevant
48+
,set_annotation_statuses(),Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
49+
,delete_annotations(),Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
50+
,upload_annotations_from_folder_to_project(),No,No,Yes,No,AWS
51+
"Annotation
52+
Classes",create_annotation_class(),Not Relevant,Not Relevant,Not Relevant,Not Relevant,Not Relevant
53+
,create_annotation_classes_from_classes_json(),Not Relevant,Not Relevant,Not Relevant,Not Relevant,AWS
54+
,search_annotation_classes(),Not Relevant,Not Relevant,Not Relevant,Not Relevant,Not Relevant
55+
,download_annotation_classes_json(),Not Relevant,Not Relevant,Not Relevant,Not Relevant,Not Relevant
56+
,delete_annotation_class(),Not Relevant,Not Relevant,Not Relevant,Not Relevant,Not Relevant
57+
Exports,prepare_export(),Yes,Yes,Yes,No,Not Relevant
58+
,download_export(),Yes,Yes,Yes,Yes,AWS
59+
,get_exports(),Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
60+
"Custom
61+
Metadata
62+
",create_custom_fields(),Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
63+
,get_custom_fields(),Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
64+
,delete_custom_fields(),Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
65+
,upload_custom_values(),Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
66+
,delete_custom_values(),Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
67+
Subsets,get_subsets(),Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
68+
,add_items_to_subset(),Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
69+
Images,download_image(),Not Relevant,Not Relevant,Not Relevant,Not Relevant,Not Relevant
70+
,download_image_annotations(),Not Relevant,Not Relevant,Not Relevant,Not Relevant,Not Relevant
71+
,upload_image_annotations(),Not Relevant,Not Relevant,Not Relevant,Not Relevant,Not Relevant
72+
,pin_image(),Not Relevant,Not Relevant,Not Relevant,Not Relevant,Not Relevant
73+
,upload_priority_scores(),Yes,Not Relevant,Not Relevant,Not Relevant,Not Relevant
74+
Team,get_team_metadata(),Not Relevant,Not Relevant,Not Relevant,Not Relevant,Not Relevant
75+
,get_integrations(),Not Relevant,Not Relevant,Not Relevant,Not Relevant,Not Relevant
76+
,invite_contributors_to_team(),Not Relevant,Not Relevant,Not Relevant,Not Relevant,Not Relevant
77+
,search_team_contributors(),Not Relevant,Not Relevant,Not Relevant,Not Relevant,Not Relevant
78+
,get_user_metadata(),Not Relevant,Not Relevant,Not Relevant,Not Relevant,Not Relevant
79+
,set_user_custom_field(),Not Relevant,Not Relevant,Not Relevant,Not Relevant,Not Relevant
80+
,list_users(),Not Relevant,Not Relevant,Not Relevant,Not Relevant,Not Relevant
81+
"Converting
82+
Annotations",import_annotation(),Not Relevant,Not Relevant,Not Relevant,Not Relevant,Not Relevant
83+
,export_annotation(),Not Relevant,Not Relevant,Not Relevant,Not Relevant,Not Relevant
84+
,convert_project_type(),Not Relevant,Not Relevant,Not Relevant,Not Relevant,Not Relevant
85+
"Working w/
86+
Annotations",validate_annotations(),Not Relevant,Not Relevant,Not Relevant,Not Relevant,Not Relevant
87+
,aggregate_annotations_as_df(),Not Relevant,Not Relevant,Not Relevant,Not Relevant,Not Relevant

‎docs/source/userguide/utilities.rst‎

Lines changed: 170 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,14 @@ Utilities
33
=========
44

55

6+
Compatibility with multimodal projects
7+
--------------------------------------
8+
9+
.. csv-table:: Support for CSV, JSON, JSONL formats
10+
:file: SDK_Functions_sheet.csv
11+
:widths: 20, 2, 15, 10, 10, 10, 25
12+
:header-rows: 1
13+
614
Converting annotation format
715
----------------------------
816

@@ -106,10 +114,171 @@ You can find more information annotation format conversion :ref:`here <ref_conve
106114
)
107115
108116
117+
Converting CSV and JSONL Formats for Annotation Management in SuperAnnotate
118+
---------------------------------------------------------------------------
119+
SuperAnnotate primarily uses the **JSONL format** for annotation import/export. However,
120+
many external tools use **CSV**, requiring users to convert between these formats for seamless data management.
121+
122+
This guide provides:
123+
124+
- CSV to JSONL conversion** for annotation uploads.
125+
- Fetching annotations from SuperAnnotate** and converting them into JSONL/CSV.
126+
- Correct metadata mappings** to ensure consistency in the annotation format.
127+
128+
129+
SuperAnnotate JSONL Schema Overview
130+
===================================
131+
Before diving into conversions, here's a breakdown of SuperAnnotate's JSONL schema:
132+
133+
.. code-block:: json
134+
135+
{
136+
"metadata": {
137+
"name": "sample_image.jpg",
138+
"item_category": { "value": "category1" },
139+
"folder_name": "dataset_folder"
140+
},
141+
"data": {
142+
"attribute1": { "value": "label1" },
143+
"attribute2": { "value": "label2" }
144+
}
145+
}
146+
147+
Key Fields:
148+
- **metadata.name** → The item's name (e.g., image file name).
149+
- **metadata.item_category** → Optional category assigned to the item.
150+
- **metadata.folder_name** → The dataset folder name (previously `_folder` in CSV).
151+
- **data** → Stores key-value pairs for attributes.
152+
153+
154+
Converting CSV to JSONL and Uploading Annotations
155+
=================================================
156+
157+
Steps to Convert CSV to JSONL
158+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
159+
160+
1. Read the **CSV file** and extract annotation fields.
161+
2. Map metadata (`_item_name`, `_item_category`, `_folder`) to **SuperAnnotate's JSONL format**.
162+
3. Convert remaining fields into JSONL **data attributes**.
163+
4. Upload the JSONL file to **SuperAnnotate using SAClient**.
164+
165+
Example Python Script:
166+
167+
.. code-block:: python
168+
169+
import csv
170+
import json
171+
from pathlib import Path
172+
from superannotate import SAClient
173+
174+
def csv_to_jsonl(csv_path, jsonl_path):
175+
"""Convert CSV annotations to JSONL format with correct mappings."""
176+
with open(csv_path, newline='', encoding='utf-8') as csv_file, open(jsonl_path, 'w', encoding='utf-8') as jsonl_file:
177+
reader = csv.DictReader(csv_file)
178+
179+
for row in reader:
180+
jsonl_entry = {
181+
"metadata": {
182+
"name": row["_item_name"],
183+
"item_category": {"value": row["_item_category"]},
184+
"folder_name": row["_folder"]
185+
},
186+
"data": {}
187+
}
188+
189+
for key, value in row.items():
190+
if key not in ["_item_name", "_item_category", "_folder"]:
191+
jsonl_entry["data"][key] = {"value": json.loads(value)}
192+
193+
json.dump(jsonl_entry, jsonl_file)
194+
jsonl_file.write('\n')
195+
196+
# Convert CSV to JSONL
197+
csv_to_jsonl("annotations.csv", "annotations.jsonl")
198+
199+
# Upload to SuperAnnotate
200+
sa = SAClient()
201+
annotations = [json.loads(line) for line in Path("annotations.jsonl").open("r", encoding="utf-8")]
202+
203+
response = sa.upload_annotations(
204+
project="project1/folder1",
205+
annotations=annotations,
206+
keep_status=True,
207+
data_spec="multimodal"
208+
)
209+
210+
211+
Fetching Annotations and Converting to JSONL/CSV
212+
================================================
213+
214+
Steps to Retrieve and Convert Annotations:
215+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
216+
217+
1. Fetch **annotations from SuperAnnotate** using `sa.get_annotations()`.
218+
2. Convert the **annotation list into JSONL format**.
219+
3. Convert the **JSONL data into CSV** for external use.
220+
221+
Python Script to Convert Annotations to JSONL:
222+
223+
.. code-block:: python
224+
225+
def convert_annotations_to_jsonl(annotations, jsonl_path):
226+
"""Convert SuperAnnotate annotations list to JSONL format."""
227+
with open(jsonl_path, 'w', encoding='utf-8') as jsonl_file:
228+
for annotation in annotations:
229+
json.dump(annotation, jsonl_file)
230+
jsonl_file.write('\n')
231+
232+
# Fetch annotations from SuperAnnotate
233+
sa = SAClient()
234+
annotations = sa.get_annotations("project", data_spec="multimodal")
235+
236+
# Convert to JSONL
237+
convert_annotations_to_jsonl(annotations, "fetched_annotations.jsonl")
238+
239+
Python Script to Convert JSONL to CSV:
240+
241+
.. code-block:: python
242+
243+
def convert_jsonl_to_csv(jsonl_path, csv_path):
244+
"""Convert JSONL file to CSV format with correct mappings."""
245+
with open(jsonl_path, 'r', encoding='utf-8') as jsonl_file, open(csv_path, 'w', newline='', encoding='utf-8') as csv_file:
246+
data = [json.loads(line) for line in jsonl_file]
247+
248+
if not data:
249+
return
250+
251+
# Extract field names from the first entry
252+
fieldnames = ["_item_name", "_item_category", "_folder"] + list(data[0]["data"].keys())
253+
254+
writer = csv.DictWriter(csv_file, fieldnames=fieldnames)
255+
writer.writeheader()
256+
257+
for entry in data:
258+
row = {
259+
"_item_name": entry["metadata"]["name"],
260+
"_item_category": entry["metadata"].get("item_category", {}).get("value"),
261+
"_folder": entry["metadata"].get("folder_name", None)
262+
}
263+
264+
for key in entry["data"]:
265+
value = entry["data"][key]
266+
row[key] = value["value"] if isinstance(value, dict) else value
267+
268+
writer.writerow(row)
269+
270+
# Convert JSONL to CSV
271+
convert_jsonl_to_csv("fetched_annotations.jsonl", "converted_annotations.csv")
272+
273+
Conclusion
274+
==========
275+
This guide provides a **seamless way to convert** annotations between CSV and JSONL formats while maintaining
276+
compatibility with **SuperAnnotate's platform**.
277+
By following these steps, users can efficiently **import, export, and manage annotation data** in their projects.
278+
109279
pandas DataFrame out of project annotations and annotation instance filtering
110280
-----------------------------------------------------------------------------
111281

112-
113282
To create a `pandas DataFrame <https://pandas.pydata.org/>`_ from project
114283
SuperAnnotate format annotations:
115284

‎src/superannotate/__init__.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
import sys
44

55

6-
__version__ = "4.4.30"
6+
__version__ = "4.4.31dev1"
77

88
os.environ.update({"sa_version": __version__})
99
sys.path.append(os.path.split(os.path.realpath(__file__))[0])

‎src/superannotate/lib/app/interface/sdk_interface.py‎

Lines changed: 8 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -505,20 +505,19 @@ def retrieve_context(
505505
) -> Tuple[bool, typing.Any]:
506506
try:
507507
for component in component_data:
508-
if (
509-
component["type"] == "webComponent"
510-
and component["id"] == component_pk
511-
):
512-
return True, component.get("context")
513-
if (
514-
component["type"] in ("group", "grid")
515-
and "children" in component
516-
):
508+
if "children" in component:
517509
found, val = retrieve_context(
518510
component["children"], component_pk
519511
)
520512
if found:
521513
return found, val
514+
if (
515+
"id" in component
516+
and component["id"] == component_pk
517+
and component["type"] == "webComponent"
518+
):
519+
return True, json.loads(component.get("context"))
520+
522521
except KeyError as e:
523522
logger.debug("Got key error:", component_data)
524523
raise e

0 commit comments

Comments
 (0)