The Pilot database

The Pilot Database

I’ve been giving some thought to this and would like to discuss the possibility transferring ownership of my database to some organization. This would be a free donation, if you will, but only if it would become an active volunteer project to expand and enhance it.

What

There are over 2000 to 3000 individual books or “bindings” in the database. That in my estimate includes yearly bindings of weekly or monthly journals, so as for journals, perhaps 10,000 individual issues. The are physically in the database, not just links.

In general, most are before 1923 which puts them into the public domain in the US. Recently Stanford University has digitized the Library of Congress Copyright renewal list which previously was insanely kept in longhand script, and was not kept up to date properly. Most magazines and journals in general circulation were never renewed but but the inability to find out if the had been was impossible, so now the Stanford database can be checked for renewal, therefore there are later issues etc. Including a complete set of CameraCraft which is in the public domain even though it goes up to the 1940’s.

How

The database structure is by directory and subdirectory. The major categories are:

100) General Manuals & Technical - All Languages\

200) Early (1839-1848) References to Photography\

300) Photography Journals\

400) Formularies\

500) Aesthetics, Pictorialism, History, Etc\

600) Photographic Processes\

700) Color Photography\

900) Roger Fenton Letters from Crimea\

1000) Photographic Optics\

1100) Miscellaneous Topics, Extracts, etc\

1200) Patents\

1250) Theses and Dissertations\

1275) City Directories\

1300) Photomechanical\

1400) General Chemistry\

1500) Early Equipment, Etc Catalogs\

1600) Photo Incunabula\

1700) AIC & Photographic Conservation\

2000) Sorting bin -- New Additions\

2100 Sortin Bin Paged\

2200) To be OCR'd\

99999) Early Cinema - To be expanded\

And these are further broken down into subgroups. This structure is key to the search engine as you can select any directory or sub-directory to limit the search. All directories and sub-directories and be turned on or off and only individual ones can be selected. There are 160 individual directories and sub-directories that can be selected for searching. Any individual publication or bound yearly can also be selected. The search engine does Regular Expression searches. It is possible that we may find a better engine if it is put online. Docfetcher does not work online.

Though the structure is biased towards early photography there are some allied groups of publications that are useful to the researcher. These include formularies, general chemistry, and city directories

Why

The question comes up: Why do all of this? They are out there already.

No, they are all over the place. Some were in the now defunct MSFT database that was in some ways duplicating Google Books, some are and libraries all over the world. Though they are in the public domain and they are often are painfully slow to access with Captcha queries etc. Yes they are available for scholarly research but there is a lot of territoriality involved in who has them in their database. In fact if in the public domain, and the vast majority are in every country, no one ‘owns” them

Much of the roadblocks placed by libraries is to prevent bots from raiding their libraries for commercial purposes or just plain online vandalism.

I estimate this database has 75% to 80% of the photography books in English, between 1839 and 1923. The the journals are in sore need of expansion, Everything is Regex searchable with the book available at search time. One can search for “Stieglitz” within 6 words of “Pizzighelli,” see snippets of the search and with a click, have the volume up in Adobe reader in seconds.

What if?

Some non-profit group took this project over? My first choice would be the Royal Photographic Society. It also appears that De Montefort University might be interested and I expect some here on the British Photo Forum have connections.

I would be available for consultation and whatever else might be needed from me.

I think the next step would be to communicate with Google books and the French National Library, Gallica, and some of the other organizations who have provided books. (The reason for this is though it is perfectly legal to download the books, Google says so but they are not happy about it. This is primarily to discourage people for downloading the books and they reselling them.) I think a good case can be made that this is a legitimate scholarly endeavor. If they could be brought into the plan and endorse it that would be quite helpful.

The Pilot database will fit on a 64 Gb thumb drive. I think we can provide a few beta copies for research and figure out a way to recover the costs of the thumb drives. I will donate a drive and the database to the RPS if they are interested, obviously Dr. Pritchard would be the one there.

There is a pdf of the publications physically in the database attached to this post.

I have spent many hours and nearly a decade building this, so one may wonder why am I doing this. A number of reasons:

I am 77 year old. Nuff said there. It needs volunteers to fill out the gaps. I already have another 50+ years of research ahead of me, 77 + 50 =?, you get the point.

And I want to see this survive me.

Cheers!

---- Dick Sullivan

The Pilot Book List.pdf

You need to be a member of British Photographic History to add comments!

Join British Photographic History

Email me when people reply –

Replies

  • Larry,

    I am not in discussion with De Montfort, just mentioning them as a possibility in case someone was lurking.

    I think you and I met a while ago at the Niepce conference at the Ransom Center. 

    And yes, I'll send a drive. I'll connect with you offline when we friend each other and you can tell me where to send it.  It is a work in progress as you can well guess but extremely useful nonetheless.

    I am using Docfetcher as an indexing tool, it's open source and does not work online however but it could be installed with something like Lucene, of which I know nothing. I retired as an IBM Systems Programmer, mainframes stuff, in 1995 and things have gotten a little advanced since than.<grin>

    --Dick Sullivan

  • Dear Dick,

    What an extraordinary project!  I am all too familiar with the role of passionate volunteer, for I have remained Editor of the Talbot Correspondence for 14 years after the official project ceased to exist.  As you know, De Montfort stepped up to the plate when Glasgow decided to pull the plug in 2005, nearly vaporizing the decades of work that went into putting transcriptions of 10,000 letters online.  Since you mention that you are in discussion with them, perhaps this will give you the lasting base that you deserve.  In the meantime, may I order one of your beta thumbdrives?  That might give me the basis for exploring other potential homes for you?  Please contact me offline with the details.  With many thanks on behalf of all photohistorians, Larry

  • Thanks for the kind words David!

    My apologies for being a little self defensive. I would like to make a few points again as I might have been a bit too wordy in my original post.

    The database as compose contains in the database itself the actual books.

    Around 3000 individual books.

    And approximately 10,000 weekly and monthly journals.

    All photographically related and organized by topic and searchable in that manner by selecting topics and subtopics individually or in groups.

    And over 750,000 pages of text. Pages in the 19th century were big with small type, tabloid size with little illustrations.

    All searchable with Regex searching, they are scanned and the search engine is open source and provided but it really belongs online with a good engine. 

    You can search for word proximity. Like find "Stieglitz" within 5 words of "Pizzighelli" or more fun "Flash Powder" within 5 words of "explosion." 

    You get an immediate list of all the hits in snippet form.

    One click then opens the actual document. 

    If you have Medelay, or other bibilo software installed, (or Acrobat, Nitro, or Foxit) you can pull quotes and pages and build a biblio automatically. (Or insert into your dissertation.)

    And much more.

    In essence, contained on the thumb drive or online database is a library of photographic books that excels any in the world. You can search issues that cannot be done online, like proximity or Regex, and get results and extracts without the answering quizzes etc.

    I want to give it away and have it offered for free. But I would like it to stay alive and be improved, not lurk in a desk drawer somewhere. 

    --Dick Sullivan

      

  • I am somewhat surprised that no one seems to have displayed interest in Richard Sullivan's submission. I have have had the privilege of examining the breadth of scope of this work on "The Pilot". The achievement is staggering, elsewhere there has been comment on the fact that the Bradford Museum only digitised 2% of the RPS Collection, in comparison Richards workis staggering, frankly I don't know how he did it seeing as he has other important work on hand. 

    This is a reference work of such magnitude, I can say there is nothing comparable to it anywhere, that not serious photo historian or researcher should be without it or at least have access to it. This latter aspect is possibly the most important, the work needs to be in the hands of a body with a significant life expectancy such as the RPS, it requires little more than a safe home from where it can be accessed by interested persons. It will not require costly upkeep as it would be web searchable.

    The matter surely calls for serious debate and practical suggestions. I would urge all those concerned with an interest in photo-historical research to open the attachment for a taste of what this database has to offer.

    David H Davison.

This reply was deleted.