Commons:Bots/Requests/Smallbot (10)
Operator: Smallman12q (talk · contributions · Statistics · Recent activity · block log · User rights log · uploads · Global account information)
Bot's tasks for which permission is being sought: To upload ~500k files from the US National Archives and Records Administration based on a database dump provided in partnership with the Digital Public Library of America.
- Example
JSON source |
---|
{
"id": "nara--1693425",
"key": "nara--1693425",
"value": {
"rev": "1-a8e9cee50a8cea9d1724c12a0d0d69e5"
},
"doc": {
"_id": "nara--1693425",
"_rev": "1-a8e9cee50a8cea9d1724c12a0d0d69e5",
"hasView": [{
"url": "http://media.nara.gov/Public_Vaults/14755_2006_001_a.jpg",
"format": "image/jpeg"
}, {
"url": "http://media.nara.gov/nwl/berryman/H-009_7-1-1928_Yes_We_Have_No_Ambitions_print.pdf",
"format": "application/pdf"
}
],
"sourceResource": {
"date": {
"begin": "1928-07-01",
"end": "1928-07-01",
"displayDate": "07/01/1928"
},
"description": "This cartoon plays off a line from a popular 1923 song (\\"
Yes,
We Have No Bananas!\\") to characterize car maker Henry Ford\'s Presidential ambitions--or lack thereof. Ford blames his busy schedule for his hesitation to jump into the \\"
Presidential contest pool,
\\" while eager supporters encourage him to \\"
come on in !\\" Berryman was correct in his prediction: Ford chose not to pursue the Presidency.",
"title": "Yes, We Have No Ambitions Today!",
"rights": "Restrictions: Unrestricted; Use status: Unrestricted",
"collection": {
"@id": "http://dp.la/api/collections/15e82b12ef89a63d03737461e2440df8",
"id": "15e82b12ef89a63d03737461e2440df8",
"title": "Records of the U.S. Senate, 1789 - 2011"
},
"stateLocatedIn": {
"name": "DC"
},
"creator": "U.S. Senate. Office of Senate Curator.\\t(? -)",
"isPartOf": "Series: Berryman Political Cartoon Collection, 1896 - 1949",
"type": "image"
},
"object": "http://media.nara.gov/Public_Vaults/14755_2006_001_t.jpg",
"ingestDate": "2013-04-11T21:27:25.187803",
"originalRecord": {
"access-restriction": {
"specific-access-restrictions": null,
"restriction-status": "Unrestricted"
},
"contributors": {
"contributor": {
"contributor-display": "Berryman, Clifford Kennedy, 1869-1949",
"contributor-record-type": "PER",
"contributor-type": "Artist",
"standard": "Y",
"num": "1",
"contributor-id": "3119843"
}
},
"hierarchy": {
"hierarchy-item": [{
"hierarchy-item-inclusive-dates": "1896 - 1949",
"hierarchy-item-id": "306080",
"hierarchy-item-lod": "Series",
"hierarchy-item-title": "Berryman Political Cartoon Collection, 1896 - 1949"
}, {
"hierarchy-item-id": "375",
"hierarchy-item-lod": "Record Group",
"hierarchy-item-title": "Records of the U.S. Senate, 1789 - 2011",
"hierarchy-item-record-group-number": "46"
}
]
},
"production-dates": {
"production-date": "07/01/1928"
},
"created-timestamp": "1/20/2013 4:36:49",
"arc-id": "1693425",
"use-restriction": {
"specific-use-restrictions": null,
"use-status": "Unrestricted"
},
"title": "Yes, We Have No Ambitions Today!",
"title-only": "Yes, We Have No Ambitions Today!",
"general-records-types": {
"general-records-type": {
"num": "1",
"general-records-type-desc": "Photographs and other Graphic Materials",
"general-records-type-id": "4237050"
}
},
"scope-content-note": "This cartoon plays off a line from a popular 1923 song (\\"
Yes,
We Have No Bananas!\\") to characterize car maker Henry Ford\'s Presidential ambitions--or lack thereof. Ford blames his busy schedule for his hesitation to jump into the \\"
Presidential contest pool,
\\" while eager supporters encourage him to \\"
come on in !\\" Berryman was correct in his prediction: Ford chose not to pursue the Presidency. ",
"parent": {
"parent-title": "Berryman Political Cartoon Collection, compiled 1896 - 1949",
"parent-lod": "Series",
"parent-id": "306080"
},
"edited-timestamp": "[g_x128_110, g_x32_443, g_x64_221, g_x2_7090, g_x16_886, g_x8_1772, g_x4_3545, b_x1_14180]",
"objects": {
"object": [{
"thumbnail-url": "http://media.nara.gov/Public_Vaults/14755_2006_001_t.jpg",
"object-sequence-number": "1",
"file-size": "579687",
"mime-type": "image/jpeg",
"num": "1",
"file-url": "http://media.nara.gov/Public_Vaults/14755_2006_001_a.jpg"
}, {
"description": "Download PDF",
"object-sequence-number": "2",
"file-size": "209895",
"mime-type": "application/pdf",
"num": "2",
"file-url": "http://media.nara.gov/nwl/berryman/H-009_7-1-1928_Yes_We_Have_No_Ambitions_print.pdf"
}
]
},
"title-date": "07/01/1928",
"subject-references": {
"subject-reference": {
"subject-type": "SRT",
"display-name": "cartoons (humorous images)",
"num": "1",
"subject-id": "4170951",
"standard": "Y"
}
},
"level-of-desc": {
"level-id": "NAVI",
"lod-display": "Item"
},
"physical-occurrences": {
"physical-occurrence": {
"media-occurrences": {
"media-occurrence": {
"num": "1",
"media-type": "Paper"
}
},
"reference-units": {
"reference-unit": {
"city": "Washington",
"fax": "202-357-5911",
"name": "Center for Legislative Archives",
"ref-id": "36",
"address2": "Room 8E, 7th and Pennsylvania Avenue NW",
"summary": "true",
"phone": "202-357-5350",
"state": "DC",
"num": "1",
"postcode": "20408",
"address1": "National Archives Building",
"mailcode": "LL",
"email": "legislative.archives@nara.gov"
}
},
"copy-status": "Preservation-Reproduction-Reference"
}
},
"creators": {
"creator": {
"creator-id": "1107050",
"standard": "Y",
"num": "1",
"creator-record-type": "ORG",
"creator-type": "Most Recent",
"creator-display": "U.S. Senate. Office of Senate Curator.\\t(? - )",
"summary": "true"
}
},
"variant-control-numbers": {
"variant-control-number": {
"mlr": "false",
"variant-number": "NWL-46-BERRYMAN-H009",
"num": "1",
"variant-type": "NAIL Control Number",
"variant-number-desc": "NWL-46-BERRYMAN-H009"
}
},
"arc-id-desc": "1693425",
"indexable-dates": {
"date-range": "[b_x16_120, b_x8_237, g_x64_30, b_x4_486, b_x16_119, g_x128_14, g_x32_59, g_x128_15, g_x16_118, g_x8_243, g_x32_60, g_x64_29, b_x2_974, b_x8_242, g_x16_121, g_x4_487]"
},
"parent-control-group": {
"parent-control-title": "Records of the U.S. Senate, 1789 - 2011",
"parent-control-lod": "Record Group",
"parent-control-id": "46"
},
"_id": "1693425"
},
"isShownAt": "http://research.archives.gov/description/1693425",
"provider": {
"@id": "http://dp.la/api/contributor/nara",
"name": "National Archives and Records Administration"
},
"@context": {
"begin": {
"@id": "dpla:dateRangeStart",
"@type": "xsd:date"
},
"@vocab": "http://purl.org/dc/terms/",
"hasView": "edm:hasView",
"name": "xsd:string",
"object": "edm:object",
"dpla": "http://dp.la/terms/",
"collection": "dpla:aggregation",
"edm": "http://www.europeana.eu/schemas/edm/",
"end": {
"@id": "dpla:end",
"@type": "xsd:date"
},
"state": "dpla:state",
"aggregatedDigitalResource": "dpla:aggregatedDigitalResource",
"coordinates": "dpla:coordinates",
"isShownAt": "edm:isShownAt",
"stateLocatedIn": "dpla:stateLocatedIn",
"sourceResource": "edm:sourceResource",
"dataProvider": "edm:dataProvider",
"originalRecord": "dpla:originalRecord",
"provider": "edm:provider",
"LCSH": "http://id.loc.gov/authorities/subjects"
},
"ingestType": "item",
"dataProvider": "Center for Legislative Archives",
"@id": "http://dp.la/api/items/02b4f072d067494f67b08d6a4100f143",
"id": "02b4f072d067494f67b08d6a4100f143"
}
}
|
- File
- Yes, We Have No Ambitions Today! - Nara - 1693425.jpg
Author |
Berryman, Clifford Kennedy, 1869-1949 |
||||||||||||||||||||||||||
Description |
English: This cartoon plays off a line from a popular 1923 song ("Yes, We Have No Bananas!") to characterize car maker Henry Ford's Presidential ambitions--or lack thereof. Ford blames his busy schedule for his hesitation to jump into the "Presidential contest pool," while eager supporters encourage him to "come on in!" Berryman was correct in his prediction: Ford chose not to pursue the Presidency. |
||||||||||||||||||||||||||
Date | 1 July 1928 | ||||||||||||||||||||||||||
Collection |
|
||||||||||||||||||||||||||
Record ID |
|
||||||||||||||||||||||||||
Source | U.S. National Archives and Records Administration | ||||||||||||||||||||||||||
Permission (Reusing this file) |
|
||||||||||||||||||||||||||
Other versions |
Please do not overwrite this file: any restoration work should be uploaded with a new name and linked in this page's "other versions=" parameter, so that this file represents the exact file found in the NARA catalog record to which it links. The metadata on this page was imported directly from NARA's catalog record; additional descriptive text may be added by Wikimedians to the template below with the "description=" parameter, but please do not modify the other fields. |
Automatic or manually assisted: Automatic
Edit type (e.g. Continuous, daily, one time run): One time
Maximum edit rate (e.g. edits per minute): 10-15, as fast as it uploads
Bot flag requested: (Y/N): No
Programming language(s): Python 3.2
Will use metadata from DPLA bulk download for NARA. The metadata is in json, and is converted formatted to the template by the bot.
Smallman12q (talk) 23:41, 8 May 2013 (UTC)
Discussion
For reference, a previous NARA batch upload was approved at Commons:Bots/Requests/US National Archives bot.Smallman12q (talk) 23:41, 8 May 2013 (UTC)
- Yeah sure, looks good to me. --Dschwen (talk) 21:21, 9 May 2013 (UTC)
- Usual suggestion: please use language template for Author/Source/Record ID fields. --EugeneZelenko (talk) 13:44, 11 May 2013 (UTC)
- Please can you put a deeplink in the "source" field, as that is where most editors will look. I tried to get the original of this example, but apparently "The Online Public Access (OPA) system will be down for maintenance from May 10 to May 25.", so we may not be able to thoroughly test this for a couple of weeks. --99of9 (talk) 13:01, 14 May 2013 (UTC)
- Yes, I recently heard some details about that as well. I'll try to keep updated on the status. Bdcousineau (talk) 14:50, 16 May 2013 (UTC)
- What kind of label is: "NWL-46-BERRYMAN-H009"? It might help to add the name of this kind of identifier. --99of9 (talk) 13:02, 14 May 2013 (UTC)
- That is an old catalog number used by NARA. It is no longer in use, but since it is in the current template used by NARA on Commons, it has been included. It most likely refers to the "NAIL" database, which was the in use prior to ARC/OPA, the current database. For a sample, see File:Football team on the field, Haskell Institute, Lawrence, Kansas, 1914 - NARA - 519149.jpg. Better removed? Bdcousineau (talk) 14:50, 16 May 2013 (UTC)
- I'd suggest leaving it in there, but having the template do nothing with it (i.e. not display it). That way we can easily reintroduce it if someone thinks it is useful later. --99of9 (talk) 15:36, 16 May 2013 (UTC)
- That is an old catalog number used by NARA. It is no longer in use, but since it is in the current template used by NARA on Commons, it has been included. It most likely refers to the "NAIL" database, which was the in use prior to ARC/OPA, the current database. For a sample, see File:Football team on the field, Haskell Institute, Lawrence, Kansas, 1914 - NARA - 519149.jpg. Better removed? Bdcousineau (talk) 14:50, 16 May 2013 (UTC)
- 500k files! Wow, this is huge, congratulations and good luck! --99of9 (talk) 13:09, 14 May 2013 (UTC)
- Great start! Since this is a large set, and since the metadata will not be perfect (never is for a transfer of this size): are you thinking of staging this? Say, a few hundred to start, then 1k, then 10k, with pauses to see what sort of cleanup is needed? --SJ+ 22:40, 15 May 2013 (UTC)
On hold-As stated at Online Public Access, access to records is suspended from the 10th to the 25th. (2 weeks is a loong roll out). Once access is restored, will do an initial batch upload of 100, 1000, then auto after that. Will also make source available once upload starts.Smallman12q (talk) 00:06, 24 May 2013 (UTC)
- Online Public Access online again. Bdcousineau (talk) 11:06, 29 May 2013 (UTC)
Can someone explain the process here? What does this have to do with DPLA (which does not host NARA images)? If you are just planning on copying the mostly low-resolution images from the catalog, I think we should slow down and concentrate on acquiring more of the high-resolution TIFF files like we did for the first mass upload. Also, with a separate mass upload based on a different set of source files, how are you planning to prevent uploading tens of thousands of duplicates? Dominic (talk) 16:37, 4 June 2013 (UTC)
- @Smallman12q, @99of9, @Dominic: What's the state of this? Is there anything we're waiting for here? odder (talk) 16:17, 16 December 2013 (UTC)
- If you ask me, this proposal wasn't very well-formed from the beginning. We already have a full-time staff member inside NARA (myself) who is working on preparing this sort of an upload, and I am working on doing it so we get the high resolution, use the full metadata from their own catalog, not DPLA, and so that it is consistent with the tens of thousands of other uploads already done. I think it is telling that my questions were never answered. Dominic (talk) 16:15, 21 December 2013 (UTC)
- Ok, unless User:Smallman12q speaks up soon, I propose that we decline this request given that User:Dominic has something superior in the works. --99of9 (talk) 03:30, 9 January 2014 (UTC)
- If you ask me, this proposal wasn't very well-formed from the beginning. We already have a full-time staff member inside NARA (myself) who is working on preparing this sort of an upload, and I am working on doing it so we get the high resolution, use the full metadata from their own catalog, not DPLA, and so that it is consistent with the tens of thousands of other uploads already done. I think it is telling that my questions were never answered. Dominic (talk) 16:15, 21 December 2013 (UTC)
Declined per above. Also, noting Smallman12q's retirement, I'd like to thank him for all his efforts in bot writing. --99of9 (talk) 03:23, 13 January 2014 (UTC)