Skip to content

Commit 498e159

Browse files
committed
Update sessions
1 parent 4564b9e commit 498e159

File tree

4 files changed

+24
-29
lines changed

4 files changed

+24
-29
lines changed

05-More-Fix-Concepts.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -204,7 +204,7 @@ colours:
204204
and you use the fix
205205

206206
```PERL
207-
add_arrayy("result") # To create a new array named result
207+
add_array("result") # To create a new array named result
208208
209209
upcase("colours[].*")
210210
append("colours[].*"," is a nice color")
@@ -228,31 +228,31 @@ If you want to only change it, under a certain condition:
228228

229229
```PERL
230230
if any_equal("colours[]","green")
231-
add_arrayy("result[]") # To create a new array named result
231+
add_array("result[]") # To create a new array named result
232232
upcase("colours[].*")
233233
append("colours[].*"," is a nice color")
234234
copy_field("colours[].*","result[].$append")
235235
end
236236
```
237237

238-
[This still transforms the all elements of an array because the conditional tests all elements not each individually.](https://metafacture.org/playground/?flux=inputFile%0A%7C+open-file%0A%7C+as-records%0A%7C+decode-yaml%0A%7C+fix%28transformationFile%29%0A%7C+encode-yaml%0A%7C+print%0A%3B&transformation=if+any_equal%28%22colours%5B%5D%22%2C%22green%22%29%0A++add_arrayy%28%22result%22%29+%23+To+create+a+new+array+named+result%0A%0A++upcase%28%22colours%5B%5D.%2A%22%29%0A++append%28%22colours%5B%5D.%2A%22%2C%22+is+a+nice+color%22%29%0A++copy_field%28%22colours%5B%5D.%2A%22%2C%22result.%24append%22%29%0Aend&data=---%0Acolours%3A%0A+-+red%0A+-+yellow%0A+-+green)
238+
[This still transforms the all elements of an array because the conditional tests all elements not each individually.](https://metafacture.org/playground/?flux=inputFile%0A%7C+open-file%0A%7C+as-records%0A%7C+decode-yaml%0A%7C+fix%28transformationFile%29%0A%7C+encode-yaml%0A%7C+print%0A%3B&transformation=if+any_equal%28%22colours%5B%5D%22%2C%22green%22%29%0A++add_array%28%22result%22%29+%23+To+create+a+new+array+named+result%0A%0A++upcase%28%22colours%5B%5D.%2A%22%29%0A++append%28%22colours%5B%5D.%2A%22%2C%22+is+a+nice+color%22%29%0A++copy_field%28%22colours%5B%5D.%2A%22%2C%22result.%24append%22%29%0Aend&data=---%0Acolours%3A%0A+-+red%0A+-+yellow%0A+-+green)
239239
To only tranform and copy the value `green` to an X you have to use the `do list`-Bind:
240240

241241
```PERL
242242
do list(path:"colours[]","var":"$i")
243243
if any_equal("$i","green")
244-
add_arrayy("result[]") # To create a new array named result
244+
add_array("result[]") # To create a new array named result
245245
upcase("$i")
246246
append("$i"," is a nice color")
247247
copy_field("$i","result[].$append")
248248
end
249249
end
250250
```
251251

252-
[See this example here in the playground.](https://metafacture.org/playground/?flux=inputFile%0A%7C+open-file%0A%7C+as-records%0A%7C+decode-yaml%0A%7C+fix%28transformationFile%29%0A%7C+encode-yaml%0A%7C+print%0A%3B&transformation=do+list%28path%3A%22colours%5B%5D%22%2C%22var%22%3A%22%24i%22%29%0A++++if+any_equal%28%22%24i%22%2C%22green%22%29%0A++++++++add_arrayy%28%22result%5B%5D%22%29+%23+To+create+a+new+array+named+result%0A++++++++upcase%28%22%24i%22%29%0A++++++++append%28%22%24i%22%2C%22+is+a+nice+color%22%29%0A++++++++copy_field%28%22%24i%22%2C%22result%5B%5D.%24append%22%29%0A++++end%0Aend&data=---%0Acolours%3A%0A+-+red%0A+-+yellow%0A+-+green)
252+
[See this example here in the playground.](https://metafacture.org/playground/?flux=inputFile%0A%7C+open-file%0A%7C+as-records%0A%7C+decode-yaml%0A%7C+fix%28transformationFile%29%0A%7C+encode-yaml%0A%7C+print%0A%3B&transformation=do+list%28path%3A%22colours%5B%5D%22%2C%22var%22%3A%22%24i%22%29%0A++++if+any_equal%28%22%24i%22%2C%22green%22%29%0A++++++++add_array%28%22result%5B%5D%22%29+%23+To+create+a+new+array+named+result%0A++++++++upcase%28%22%24i%22%29%0A++++++++append%28%22%24i%22%2C%22+is+a+nice+color%22%29%0A++++++++copy_field%28%22%24i%22%2C%22result%5B%5D.%24append%22%29%0A++++end%0Aend&data=---%0Acolours%3A%0A+-+red%0A+-+yellow%0A+-+green)
253253

254254
TO BE CONTINUED ...
255255

256-
TODO: Add Excercises.
256+
TODO: Add more Excercises.
257257

258258
Next lesson: [06 Metafacture CLI](./06_MetafactureCLI.md)

07_Processing_MARC.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -291,12 +291,12 @@ Step 1, create a fix file `transformationFile.fix` containing:
291291

292292
```PERL
293293
copy_field("001","id")
294-
add_arrayy("title")
294+
add_array("title")
295295
do list(path: "245??.?","var":"$i")
296296
copy_field("$i","title.$append")
297297
end
298298
join_field(title," ")
299-
add_arrayy("isbn")
299+
add_array("isbn")
300300
do list(path: "020??","var":"$i")
301301
copy_field("$i.a",isbn.$append)
302302
end
@@ -320,7 +320,7 @@ HINT: Sometimes it makes sense to create an empty array by `add_array` or an emp
320320
321321
```
322322

323-
[See it in the Playground here.](https://metafacture.org/playground/?flux=%22https%3A//raw.githubusercontent.com/metafacture/metafacture-tutorial/main/data/sample.mrc%22%0A%7C+open-http%0A%7C+as-lines%0A%7C+decode-marc21%0A%7C+fix%28transformationFile%29%0A%7C+encode-csv%28includeheader%3D%22true%22%29%0A%7C+print%0A%3B%0A%0A&transformation=copy_field%28%22001%22%2C%22id%22%29%0Aadd_arrayy%28%22title%22%29%0Ado+list%28path%3A+%22245%3F%3F.%3F%22%2C%22var%22%3A%22%24i%22%29%0A++copy_field%28%22%24i%22%2C%22title.%24append%22%29%0Aend%0Ajoin_field%28title%2C%22+%22%29%0Aadd_arrayy%28%22isbn%22%29%0Ado+list%28path%3A+%22020%3F%3F.a%22%2C%22var%22%3A%22%24i%22%29%0A++copy_field%28%22%24i%22%2Cisbn.%24append%29%0Aend%0Ajoin_field%28isbn%2C%22%2C%22%29%0Aretain%28%22id%22%2C%22title%22%2C%22isbn%22%29)
323+
[See it in the Playground here.](https://metafacture.org/playground/?flux=%22https%3A//raw.githubusercontent.com/metafacture/metafacture-tutorial/main/data/sample.mrc%22%0A%7C+open-http%0A%7C+as-lines%0A%7C+decode-marc21%0A%7C+fix%28transformationFile%29%0A%7C+encode-csv%28includeheader%3D%22true%22%29%0A%7C+print%0A%3B%0A%0A&transformation=copy_field%28%22001%22%2C%22id%22%29%0Aadd_array%28%22title%22%29%0Ado+list%28path%3A+%22245%3F%3F.%3F%22%2C%22var%22%3A%22%24i%22%29%0A++copy_field%28%22%24i%22%2C%22title.%24append%22%29%0Aend%0Ajoin_field%28title%2C%22+%22%29%0Aadd_array%28%22isbn%22%29%0Ado+list%28path%3A+%22020%3F%3F.a%22%2C%22var%22%3A%22%24i%22%29%0A++copy_field%28%22%24i%22%2Cisbn.%24append%29%0Aend%0Ajoin_field%28isbn%2C%22%2C%22%29%0Aretain%28%22id%22%2C%22title%22%2C%22isbn%22%29)
324324

325325

326326
You will see this as output:
@@ -339,7 +339,7 @@ You will see this as output:
339339
"1080278184","Renfro Valley Kentucky Rainer H. Schmeissner",""
340340
```
341341

342-
In the fix above we mapped the 245-field to the title, and iterated over every subfield with the help of the list-bind and the `?`- wildcard. The ISBN is in the 020-field. Because MARC records can contain one or more 020 fields we created an isbn array with add_arrayy and added the values using the isbn.$append syntax. Next we turned the isbn array back into a comma separated string using the join_field fix. As last step we deleted all the fields we didn’t need in the output with the `retain` syntax.
342+
In the fix above we mapped the 245-field to the title, and iterated over every subfield with the help of the list-bind and the `?`- wildcard. The ISBN is in the 020-field. Because MARC records can contain one or more 020 fields we created an isbn array with add_array and added the values using the isbn.$append syntax. Next we turned the isbn array back into a comma separated string using the join_field fix. As last step we deleted all the fields we didn’t need in the output with the `retain` syntax.
343343

344344
Different versions of MARC-Serialization need different workflows: e.g. h[ere see an example of Aseq-Marc Files that are transformed to marcxml.](https://test.metafacture.org/playground/?flux=%22https%3A//raw.githubusercontent.com/LibreCat/Catmandu-MARC/dev/t/rug01.aleph%22%0A%7C+open-http%0A%7C+as-lines%0A%7C+decode-aseq%0A%7C+merge-same-ids%0A%7C+encode-marcxml%0A%7C+print%0A%3B)
345345

11_MARC_to_Dublin_Core.md

Lines changed: 13 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
1-
# TODO : Lesson 11 : From MARC to Dublin Core as loud JSON-LD
2-
TODO: Use better example. https://github.com/metafacture/metafacture-examples/blob/master/Swissbib-Extensions/MARC-CSV/
1+
# Lesson 11 : From MARC to Dublin Core as loud JSON-LD
2+
TODO: Use better example. But the following is missing isbns: https://github.com/metafacture/metafacture-examples/blob/master/Swissbib-Extensions/MARC-CSV/
33

44
Today we will look a bit further into MARC processing with Metafacture. We already saw a bit of MARC processing in and today we will show you how to transform MARC records into Dublin Core and providing the data as linked open usable data.
55

@@ -9,22 +9,22 @@ And type into this textfile the following fixes:
99

1010
```PERL
1111
copy_field("245??.a","title")
12-
add_arrayy("creator[]")
12+
add_array("creator[]")
1313
copy_field("100??.a","creator[].$append")
1414
copy_field("700??.a","creator[].$append")
1515
copy_field("260??.c","date")
1616
copy_field("260??.b","publisher")
1717

18-
add_arrayy("isbn[]")
18+
add_array("isbn[]")
1919
do list(path:"020??","var":"$i")
2020
copy_field("$i.a","isbn[].$append")
2121
end
22-
add_arrayy("issn[]")
22+
add_array("issn[]")
2323
do list(path:"022??","var":"$i")
2424
copy_field("$i.a","issn[].$append")
2525
end
2626

27-
add_arrayy("subject[]")
27+
add_array("subject[]")
2828
do list(path:"650??","var":"$i")
2929
copy_field("$i.a","subject[].$append")
3030
end
@@ -87,10 +87,10 @@ Congratulations, you’ve created your first mapping file to transform library d
8787
Below you’ll find a complete example. You can read more about our Fix language online.
8888

8989
```PERL
90-
add_arrayy("title")
90+
add_array("title")
9191
copy_field("245??.?","title.$append")
9292
join_field("title", " ")
93-
add_arrayy("creator[]")
93+
add_array("creator[]")
9494
copy_field("100??.a","creator[].$append")
9595
copy_field("700??.a","creator[].$append")
9696
copy_field("260??.c","date")
@@ -100,19 +100,19 @@ replace_all("publisher",",$","")
100100
101101
add_field("type","BibliographicResource")
102102
103-
add_arrayy("isbn[]")
103+
add_array("isbn[]")
104104
do list(path:"020??","var":"$i")
105105
copy_field("$i.a","isbn[].$append")
106106
end
107107
replace_all("isbn.*"," .","")
108108
109-
add_arrayy("issn[]")
109+
add_array("issn[]")
110110
do list(path:"022??","var":"$i")
111111
copy_field("$i.a","issn[].$append")
112112
end
113113
replace_all("issn.*"," .","")
114114
115-
add_arrayy("subject[]")
115+
add_array("subject[]")
116116
do list(path:"650??","var":"$i")
117117
copy_field("$i.a","subject[].$append")
118118
end
@@ -135,11 +135,6 @@ add_field("@context.issn","http://purl.org/ontology/bibo/issn")
135135
```
136136
137137
The result should look like this:
138-
https://metafacture.org/playground/?flux=%22https%3A//raw.githubusercontent.com/metafacture/metafacture-core/master/metafacture-runner/src/main/dist/examples/read/marc21/10.marc21%22%0A%7C+open-http%0A%7C+as-lines%0A%7C+decode-marc21%0A%7C+fix%28transformationFile%29%0A%7C+encode-json%28prettyPrinting%3D%22true%22%29%0A%7C+print%0A%3B&transformation=add_arrayy%28%22title%22%29%0Acopy_field%28%22245%3F%3F.%3F%22%2C%22title.%24append%22%29%0Ajoin_field%28%22title%22%2C+%22+%22%29%0Aadd_arrayy%28%22creator%5B%5D%22%29%0Acopy_field%28%22100%3F%3F.a%22%2C%22creator%5B%5D.%24append%22%29%0Acopy_field%28%22700%3F%3F.a%22%2C%22creator%5B%5D.%24append%22%29%0Acopy_field%28%22260%3F%3F.c%22%2C%22date%22%29%0Areplace_all%28%22date%22%2C%22\\D%2B%22%2C%22%22%29%0Acopy_field%28%22260%3F%3F.b%22%2C%22publisher%22%29%0Areplace_all%28%22publisher%22%2C%22%2C%24%22%2C%22%22%29%0A%0Aadd_field%28%22type%22%2C%22BibliographicResource%22%29%0A%0Aadd_arrayy%28%22isbn%5B%5D%22%29%0Ado+list%28path%3A%22020%3F%3F%22%2C%22var%22%3A%22%24i%22%29%0A++++copy_field%28%22%24i.a%22%2C%22isbn%5B%5D.%24append%22%29%0Aend%0Areplace_all%28%22isbn.%2A%22%2C%22+.%22%2C%22%22%29%0A%0Aadd_arrayy%28%22isbn%5B%5D%22%29%0Ado+list%28path%3A%22022%3F%3F%22%2C%22var%22%3A%22%24i%22%29%0A++++copy_field%28%22%24i.a%22%2C%22issn%5B%5D.%24append%22%29%0Aend%0Areplace_all%28%22issn.%2A%22%2C%22+.%22%2C%22%22%29%0A%0Aadd_arrayy%28%22subject%5B%5D%22%29%0Ado+list%28path%3A%22650%3F%3F%22%2C%22var%22%3A%22%24i%22%29%0A++++copy_field%28%22%24i.a%22%2C%22subject%5B%5D.%24append%22%29%0Aend%0A%0Aadd_field%28%22@context.title%22%2C%22http%3A//purl.org/dc/terms/title%22%29%0Aadd_field%28%22@context.creator%22%2C%22http%3A//purl.org/dc/elements/1.1/creator%22%29%0Aadd_field%28%22@context.date%22%2C%22http%3A//purl.org/dc/elements/1.1/date%22%29%0Aadd_field%28%22@context.publisher%22%2C%22http%3A//purl.org/dc/elements/1.1/publisher%22%29%0Aadd_field%28%22@context.subject%22%2C%22http%3A//purl.org/dc/elements/1.1/subject%22%29%0Aadd_field%28%22@context.isbn%22%2C%22http%3A//purl.org/ontology/bibo/isbn%22%29%0Aadd_field%28%22@context.issn%22%2C%22http%3A//purl.org/ontology/bibo/issn%22%29%0A%0Aretain%28%22@context%22%2C%22title%22%2C%22creator%5B%5D%22%2C%22date%22%2C%22publisher%22%2C%22isbn%5B%5D%22%2C%22issn%5B%5D%22%2C%22subject%5B%5D%22%29
138+
https://metafacture.org/playground/?flux=%22https%3A//raw.githubusercontent.com/metafacture/metafacture-core/master/metafacture-runner/src/main/dist/examples/read/marc21/10.marc21%22%0A%7C+open-http%0A%7C+as-lines%0A%7C+decode-marc21%0A%7C+fix%28transformationFile%29%0A%7C+encode-json%28prettyPrinting%3D%22true%22%29%0A%7C+print%0A%3B&transformation=add_array%28%22title%22%29%0Acopy_field%28%22245%3F%3F.%3F%22%2C%22title.%24append%22%29%0Ajoin_field%28%22title%22%2C+%22+%22%29%0Aadd_array%28%22creator%5B%5D%22%29%0Acopy_field%28%22100%3F%3F.a%22%2C%22creator%5B%5D.%24append%22%29%0Acopy_field%28%22700%3F%3F.a%22%2C%22creator%5B%5D.%24append%22%29%0Acopy_field%28%22260%3F%3F.c%22%2C%22date%22%29%0Areplace_all%28%22date%22%2C%22\\D%2B%22%2C%22%22%29%0Acopy_field%28%22260%3F%3F.b%22%2C%22publisher%22%29%0Areplace_all%28%22publisher%22%2C%22%2C%24%22%2C%22%22%29%0A%0Aadd_field%28%22type%22%2C%22BibliographicResource%22%29%0A%0Aadd_array%28%22isbn%5B%5D%22%29%0Ado+list%28path%3A%22020%3F%3F%22%2C%22var%22%3A%22%24i%22%29%0A++++copy_field%28%22%24i.a%22%2C%22isbn%5B%5D.%24append%22%29%0Aend%0Areplace_all%28%22isbn.%2A%22%2C%22+.%22%2C%22%22%29%0A%0Aadd_array%28%22isbn%5B%5D%22%29%0Ado+list%28path%3A%22022%3F%3F%22%2C%22var%22%3A%22%24i%22%29%0A++++copy_field%28%22%24i.a%22%2C%22issn%5B%5D.%24append%22%29%0Aend%0Areplace_all%28%22issn.%2A%22%2C%22+.%22%2C%22%22%29%0A%0Aadd_array%28%22subject%5B%5D%22%29%0Ado+list%28path%3A%22650%3F%3F%22%2C%22var%22%3A%22%24i%22%29%0A++++copy_field%28%22%24i.a%22%2C%22subject%5B%5D.%24append%22%29%0Aend%0A%0Aadd_field%28%22@context.title%22%2C%22http%3A//purl.org/dc/terms/title%22%29%0Aadd_field%28%22@context.creator%22%2C%22http%3A//purl.org/dc/elements/1.1/creator%22%29%0Aadd_field%28%22@context.date%22%2C%22http%3A//purl.org/dc/elements/1.1/date%22%29%0Aadd_field%28%22@context.publisher%22%2C%22http%3A//purl.org/dc/elements/1.1/publisher%22%29%0Aadd_field%28%22@context.subject%22%2C%22http%3A//purl.org/dc/elements/1.1/subject%22%29%0Aadd_field%28%22@context.isbn%22%2C%22http%3A//purl.org/ontology/bibo/isbn%22%29%0Aadd_field%28%22@context.issn%22%2C%22http%3A//purl.org/ontology/bibo/issn%22%29%0A%0Aretain%28%22@context%22%2C%22title%22%2C%22creator%5B%5D%22%2C%22date%22%2C%22publisher%22%2C%22isbn%5B%5D%22%2C%22issn%5B%5D%22%2C%22subject%5B%5D%22%29
139139
140-
141-
> TODO:
142-
> Create the metafacture workflow to transform Marc to JSON.
143-
> Map the marc data to DC.
144-
> Also add an context for creating JSON LD.
145-
> Add excersise.
140+
Excercice: Add additional dublincore elements that you would like to add.

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ Lesson plan:
3030

3131
[07 Processing MARC](./07_Processing_MARC.md)
3232

33-
[08 Harvest data with OAI-PMH](./08_Harvest_data_with_OAI-PMH.md) (TODO)
33+
[08 Harvest data with OAI-PMH](./08_Harvest_data_with_OAI-PMH.md)
3434

3535
[09 Working with CSV and TSV](./09_Working_with_CSV.md)
3636

0 commit comments

Comments
 (0)