Seven Tips for Using the Trading Consequences Database and Visualization Tools

The strengths of the Trading Consequences relational database and visualization tools lie in the information they convey about the geospatial history of particular commodities. There are a variety of ways to explore the database and visualizations. But for someone unfamiliar with the tools and the information they make available, Trading Consequences can be a bit overwhelming at first. In order to help first time users and non-specialists, here’s a list of tips to help you use the tools.

Start with a commodity, not a location. Interesting information is available for specific places, but is much more limited than the information available for specific commodities. Using location searches can be useful, but only after some initial work has been done to explore a commodity.

Tip #1 – Do a Wikipedia search

It helps to have a basic familiarity with the history of the commodity you’re searching. In most cases, if you don’t already know the basics of your commodity’s history, a Wikipedia search will provide more than enough information to get the background (where it came from, what it was used for, where it was consumed/bought).

Tip #2 – Start with the big picture

After choosing a commodity to explore, start with the Location Cloud Visualization search bar. For example, here’s the result if Ivory is entered.

Ivory main

Every location listed in the cloud is a place associated with Ivory in the entire corpus of documents. Each column represents a different decade between 1800 and 1910 (left to right). Locations are listed in alphabetical order, with the largest font representing the most frequently mentioned locations for that decade.

In this case, immediately obvious is the large number of mentions related to Havana. Your initial inclination might be to pursue what appears to be an abnormal result. But hold off on that for now!

Tip #3 – Isolate individual continents

Apart from the largest words in the cloud, it’s hard to make sense of all the information on display. To begin with, use the world map at the top of the screen to isolate the results for individual continents. Hovering the mouse over different continents will highlight locations from each continent in different colours. For example, hovering the mouse over Europe will highlight in green all the places in Europe associated with Ivory in the documents. Similarly, hovering the mouse over Africa will highlight in purple all the places in Africa associated with Ivory in the documents. Ivory Africa (hover)

Hovering the mouse over a continent will also display the total number of mentions of Ivory associated with that continent in a small text box next to the mouse (e.g. the total number of mentions for Africa is 3,137).

Actually clicking on an individual continent will control the location cloud to display only the places from that continent. For example, clicking on Africa will eliminate all locations other than places in Africa.

Ivory Africa

This will make it easier to search for patterns in the location cloud. Once you’ve finished exploring one continent, simply click on the continent again to disengage the control and return to the full location cloud. You can click on any number of continents to control for different combinations (clicking on both Europe and Africa, for example, will eliminate all other continents and display locations for places in Europe and Africa alone).

Try isolating for each continent individually as well as different combinations of continents.

Tip #4 – Isolate for specific places

It may still be difficult to identify potential patterns from the location cloud for an individual continent. Hovering the mouse over particular location names in the cloud will allow you to isolate spatial and temporal patterns. Moreover, hovering the mouse over location names in the cloud reveals the number of mentions of Ivory associated with that specific place in that decade. For example, hovering the mouse over Mozambique in the location cloud reveals there are 42 mentions of Ivory during the 1850s. More importantly, most mentions of Ivory associated with Mozambique occur during the 1840s and 1850s. Ivory Mozambique (hover)

Similarly, hovering the mouse over Zanzibar (off the coast of Tanzania) reveals there are 67 mentions of Ivory during the 1900s and that most mentions of Ivory associated with Zanzibar occur between the 1870s and the 1910s.

Ivory Zanzibar (hover)

This suggests that the ivory trade was important in Mozambique during the early nineteenth century, but that trade through Zanzibar became more important toward the end of the century.

Actually clicking on a specific location in a particular decade brings up a window display of the timeline distribution of mentions of Ivory associated with that place (normalized by all commodity/location mentions from 1800-1920), as well as a short list of selected sentences from documents that feature Ivory and that location during the decade chosen.

Ivory Zanzibar (distribution)

This visualization will give you a clearer sense of the changing trends in mentions of Ivory associated with that location (in this case Zanzibar).

Tip #5 – Explore excerpts from the documents

Once you have identified what you think are likely significant trends, it makes sense to follow up your hypothesis with a quick look through excerpts from the primary source documents in which the mentions occurred. Included in the window that pops up when you click on a specific place in one of the location cloud columns is a link to a list of documents that feature Ivory and the chosen location.

For example, clicking on the Mozambique location in 1850s column of the location cloud brings up a window with a distribution of mentions and a list of selected sentences. Clicking on the link to the full document list in the pop up window opens a new page with a list of primary source documents. By clicking on the small box next to the name of the document, you can explore the sentences that feature Ivory and Mozambique in that document. The larger the box, the more mentions that document includes.

Ivory Mozambique (docs list)

Reading through these documents allows you to determine a more accurate picture of the nature of the relationship between Ivory and Mozambique during the 1850s.

Tip #6 – Explore the other end of the commodity chain

If you started with the production end of the commodity chain (i.e. Ivory comes from Africa), try exploring the consumption side of the commodity chain (i.e. places where Ivory ended up).

Instead of isolating Africa, try isolating Europe or North America. Look for similar types of patterns using the same methods described above and then follow up with the primary sources by clicking on promising-looking specific locations on the location cloud.

For example, isolating Europe in the location cloud reveals a high number of mentions associated with Lisbon (Portugal) earlier in the nineteenth century,

Ivory Lisbon (distribution)

 

followed by a rise in the number of mentions associated with Germany toward the end of the century.

Ivory Germany (distribution)

During the nineteenth century, Mozambique was a Portuguese colony and Germany held imperial interests (along with Britain) in Zanzibar. The patterns of mentions of Ivory in Europe, therefore, suggest a strong correlation with the patterns of mentions of Ivory associated with Africa.

Indeed a quick reading of excerpts from the documents confirms this to be the case.

In this list of documents from the 1850s, Lisbon is mentioned in relation to Mozambique and Ivory several times.

Ivory Lisbon (docs list)

Similarly, an 1888 document dealing with relations between Germany and Zanzibar mentions Ivory as a commodity key to peace negotiations.

Ivory Germany (docs list)

Tip #7 – Cross-reference with a full corpus commodity search

A separate Commodity search bar allows you to cross-reference the information obtained from your exploration of the commodity-location patterns in the Location Cloud Visualization.

For example, here’s the result if Ivory is entered.

Ivory commodity search

The list on the right contains all the documents in which Ivory is mentioned (use the ‘next’ button at the bottom to advance to the next page). This list can be narrowed down into ten year blocks by clicking on a specific decade at the bottom left (the number in brackets next to each decade denotes the number of mentions of Ivory in that decade).

Hopefully these tips will help get you started using the database and visualization tools. Trading Consequences is a great resource with a lot to offer anyone interested in global commodity flows during the nineteenth century. To use these tools effectively, however, requires several sessions in between which users must be spend time reading through the literature to confirm results and determine new avenues of enquiry to guide further use of Trading Consequences.

Official Launch of Trading Consequences!

Today we are delighted to officially announce the launch of Trading Consequences!

Over the course of the last two years the project team have been hard at work to use text mining, traditional and innovative historical research methods, and visualization techniques, to turn digitized nineteenth century papers and trading records (and their OCR’d text) into a unique database of commodities and engaging visualization and search interfaces to explore that data.

Today we launch the database, searches and visualization tools alongside the Trading Consequences White Paper, which charts our work on the project including technical approaches, some of the challenges we faced, and what and how we have achieved during the project. The White Paper also discusses, in detail, how we built the tools we are launching today and is therefore an essential point of reference for those wanting to better understand how data is presented in our interfaces, how these interfaces came to be, and how you might best use and interpret the data shared in these resources in your own historical research.

Find the Trading Consequences searches, visualizations and code via the panel on the top right hand side of the project website (outlined in orange).

Find the Trading Consequences searches, visualizations and code via the panel on the top right hand side of the project website (outlined in orange).

There are four ways to explore the Trading Consequences database:

  1. Commodity Search. This performs a search of the database table of unique commodities, for commodities beginning with the search term entered. The returned list of commodities is sorted by two criteria (1) whether the commodity is a “commodity concept” (where any one of several unique names known to be used for the same commodity returns aggregated data for that commodity); or (2) alphabetically. Read more here.
  2. Location SearchThis performs a search of the database table of unique locations, for locations beginning with the search term entered. The returned list of locations is sorted by the frequency that the search term is mentioned within the historical documents. Selecting a location displays: information about the location such as which country it is within, population etc; A map highlighting the location with a map marker; A list of historical documents and an indication of how many times the selected location is mentioned within each document. Read more here.
  3. Location Cloud Visualization. This shows the relation between a selected commodity and its related location. The visualization is based on over 170000 documents from digital historical archives (see list of archives below).The purpose of the visualization is to provide a general overview of how the importance of location mentions in relation to a particular commodity changed between 1800 and 1920. Read more here.
  4. Interlinked Visualization. This provides a general overview of how commodities were discussed between 1750 and 1950 along geographic and temporal dimensions. They provide an overview of commodity and location mentions extracted from 179000 historic documents (extracted from the digital archive listed below). Read more here.

Please do try out these tools (please note that the two visualizations will only work with newer versions of the Chrome Browserand let us know what you think – we would love to know what other information or support might be useful, what feedback you have for the project team, how you think you might be able to use these tools in your own research.

Image of the Start page of the Interlinked Visualization.

Start page of the Interlinked Visualization.

We are also very pleased to announce that we are sharing some of the code and resources behind Trading Consequences via GitHub. This includes a range of Lexical Resources that we think historians and those undertaking historical text mining in related areas, may find particularly useful: the base lexicon of commodities created by hand for this project; the Trading Consequences SKOS ontology; and an aggregated gazeteer of ports and cities with ports.

Bea Alex shares text mining progress with the team at an early Trading Consequences meeting.

Bea Alex shares text mining progress with the team at an early Trading Consequences meeting.

Acknowledgements

The Trading Consequences team would like to acknowledge and thank the project partners, funders and data providers that have made this work possible. We would particularly like to thank the Digging Into Data Challenge, and the international partners and funders of DiD, for making this fun, challenging and highly collaborative transatlantic project possible. We have hugely enjoyed working together and we have learned a great deal from the interdisciplinary and international exchanges that has been so central to to this project.

We would also like to extend our thanks to all of those who have supported the project over the last few years with help, advice, opportunities to present and share our work, publicity for events and blog posts. Most of all we would like to thank all of those members of the historical research community who generously gave their time and perspectives to our historians, to our text mining experts, and particularly to our visualization experts to help us ensure that what we have created in this project meets genuine research needs and may have application in a range of historical research contexts.

Image of the Trading Consequences Project Team at our original kick off meeting.

Image of the Trading Consequences Project Team at our original kick off meeting.

What next?
Trading Consequences does not come to an end with this launch. Now that the search and visualization tools are live – and open for anyone to use freely on the web – our historians Professor Colin Coates (York University, Canada) and Dr Jim Clifford (University of Saskatchewan) will be continuing their research. We will continue to share their findings on historical trading patterns, and environmental history, via the Trading Consequences blog.

Over the coming months we will be continuing to update our publications page with the latest research and dissemination associated with the project, and we will also be sharing additional resources associated with the project via GitHub, so please do continue to keep an eye on this website for key updates and links to resources.

We value and welcome your feedback on the visualizations, search interfaces, the database, or any other aspect of the project, website or White Paper at any point. Indeed, if you do find Trading Consequences useful in your own research we would particularly encourage you to get in touch with us (via the comments here, or via Twitter) and consider writing a guest post for the blog. We also welcome mentions of the project or website in your own publications and we are happy to help you to publicize these.

Image of Testing and feedback at CHESS'13.

Testing and feedback at CHESS’13.

Explore Trading Consequences

Comparing Apples with Oranges

This Friday we will officially launch Trading Consequences this Friday (21st March), with publication of our White Paper and the launch of our visualization and search tools. Ahead of the launch we wanted to give you some idea of what you will be able to access, what you might want to view and what you might want to compare with these new historical research tools. Professor Colin Coates has been exploring the possibilities… 

The “Trading Consequences” website literally allows us to compare apples and oranges.  Both fruits became the objects of substantial international trade in the nineteenth century, as in the right conditions they can remain edible despite being shipped great distances.

Screen shot of a visualisation of Apple Trades

They are complementary fruits in many ways, as apples are grown in temperate climates whilst oranges prefer warmer conditions.  They may overlap geographically, but typically we associate different parts of the world with each fruit.  In the context of the British world, apples grew in the United Kingdom, of course, but they also came from Canada, New Zealand and the United States, among other locations.  Oranges from places like Spain, Florida or Latin America entered the United Kingdom in the nineteenth century.  The two maps which result from entering “apple” and “orange” into the database show, at a glance, how oranges appeared more often in reference to warmer zones than apples.

Screen shot of a visualisation of Orange Trades

The chronological distribution of commodity mentions was roughly similar in both cases.  Increased attention from 1880 to 1900 reflects in part the expansion of the documentation in that period, but it likely also reflected growth in trade and consumption.  Historian James Murton has pointed out that regular trade in apples developed from Canada to Great Britain in the 1880s, focused primarily in Nova Scotia.  On average, one million bushels of apples reached British markets (Murton, 2012).

In contrast, both apples and oranges show sudden spikes in the 1830s, for entirely different reasons.  The spike for apples points the researcher to a useful “Report from the Selection Committee on the Fresh Fruit Trade” in 1839.  But the mid-1830s spike in oranges points instead to the activities of Orange Lodges in Ireland.  The other visualisation shows this anomaly even more clearly, as IRELAND takes on a prominence in related geographical terms in the 1830s that it did not occupy afterwards.

Screenshot of Visualisation looking at trades in the 1830s

This project entailed teaching computers to read as an historian might, and there are distinct advantages to being able to deal with such a wide range of documentation.  However, all historians must be critical of the sources we use. The visualisations in “Trading Consequences” point towards useful sources for further study, and to suggest that historian may wish to consider some regions in their analysis.  The importance of the United States in the discussions about apples is noteworthy, for instance.  Australia has a large number of mentions of oranges, though it is important to note that a small city boasts the same name and could account for part of the number.  (Interestingly enough, Orange, New South Wales, did not grow many oranges according to the Australian Atlas 2006! But it does have apples.)

"Fruit" by Flickr user Garry Knight / garryknight

“Fruit” by Flickr user Garry Knight / garryknight

The increase in mentions of both apples and oranges from the 1880s on may reflect improving living standards in Britain in that period.  Britain’s decision to adopt free trade had led to an increase in a wide variety of imported foodstuffs (Darwin, 2009).  As the heightened attention to both apples and oranges probably shows, these fruits were part of that movement.

The “Trading Consequences” visualisations show some instructive comparisons, some that may point to different ways to conceive of trade in these resources, and others which illustrate the care with which researchers should approach results.

References

  • John Darwin, The Empire Project: The Rise and Fall of the British World-System, 1830-1970 (Cambridge: Cambridge University Press, 2009)
  •  James Murton, “John Bull and Sons: The Empire Marketing Board and the Creation of a British Imperial Food System” in Franca Iacovetta et al., eds., Edible Histories, Cultural Politics: Towards a Canadian Food History (Toronto: University of Toronto Press, 2012), 234-35.
  • New South Wales Government, Agriculture – Fruit and Vegetables in the Atlas of New South Wales, Available from: http://www.atlas.nsw.gov.au/public/nsw/home/topic/article/agriculture-fruit-and-vegetables.html

10 things we learned at the Trading Consequences project meeting…

On Thursday 17th and Friday 18th May we held a Trading Consequences project meeting in Edinburgh where the whole team finally got to meet each other after months of virtual meetings. Here are the 10 awesome things we found out…

  1. Visualisation isn’t about pretty pictures it’s about insight. Take for example the  London Underground map and a New York Subway map… you will see some seriously different stylings (you can see both in Aaron’s presentation here). The London Underground Map is all about key points on the routes, the map isn’t a literal representation of distance but a conceptual take on London’s origins as a network of villages. In New York, where residents are used to walking above ground and are particularly used to the grid system for roads the map reflects this in order to make it easier to conceptualise the combination of Subway and walking routes. And that’s the key thing… visualisations are about representing different world views, different conceptions of information, specific mental maps of the data. A good visualisation reflects a particular world view rather than trying to loyally mirror reality.
  2. Image of a banana

    Moved banana by Flickr user ungard | dave ungar

    Yes, we have no bananas! Well, actually, we might have some bananas today but in London in 1905 did you know that you were allowed to steal bananas if they were brown or blackened? There is an oral history description of being allowed to steal these bananas as they couldn’t be sold. So, can we find evidence to back this up? If we are going to then we need to leave as much information in the ontology we are building to ensure we can find and access that sort of detail. Of course we know what we want to look for here – banana-bread ready fruit is a bit of a known unknown – but what about the things we don’t know about yet? The unknown unknowns we may want to find in the future? Not being able to find something in the data we have gathered doesn’t necessarily mean it’s not there, it just means we can’t confirm that it’s there.
  3. The 19th Century take on “animal, vegetable, or mineral?” was “from the sea“, “from the farm“, or “from the forest”?  This is all about ontologies again… So what is an ontology? Well it’s a way to understand the world, a conceptual model that allows you to structure, sort, classify, connect and understand each item within its immediate and wider context. In an era of trading raw materials and early manufactured items “from the sea” made sense, “from the farm” added useful context… similarly we might be used to understanding trees by their genus but historically qualities such as whether it can be sawn or hewn were important classifications. We’ve been thinking about this since the meeting and you can read about some of the issues around ontologies on Ewan’s blog.
  4. Image of artificial eyes

    Eyes (NOT FOR SALE) by Flickr User fumikaharukaze | Fumika Harukaze

    The eyes have it… and that can be a real problem as us humans are quite a lot better built for reading visual information than machines. When we are looking at sources for Trading Consequences we are seeing digitised materials that have been scanned then OCRed (put through Optical Character Recognition). Printing presses used to be pretty quirky – the letter “a” might look squiffy in every print, or a mark might appear on every page, ink may have smudged, etc. Scanning and OCR technology might look much more high tech but they too have quirks – digital cameras and scanners get better all the time and OCR engines improve each year… that means materials we are working with that were digitised years back look noticibly different from those that have been recently scanned and OCRed. That can be pretty challenging… and then we get to the many tables of traded goods. The human may see a very attractive pattern of columns and rows but the computer just doesn’t see it that easily and we have to try to guide it to read the data in so that it makes sense to the machine, to us humans, and that it reflects what was in the original document.
  5. Image of turkey red cotton

    "Turkey red floral patterns." by the National Museum of Scotland's Feastbowl Blog (click through to read a full post on Turkey Red)

    Wild turkey and rubber demands…. Turkey Red is a type of dyed cotton – named after the place not the bird – which was exported in huge amounts, much of it from Aberdeen But Turkey Red was a complicated and expensive die to make and the process was incompatible with the new textile printing processes that were emerging. There was a shift from natural dyes to synthetic materials and demand for Turkey Red plummeted. The project team has been in discussion with Edinburgh University’s Stana Nenadic and her Colouring the Nation project, which specifically looks at the history of Turkey Red. However, this is just one great example of changes in society being echoed by the consequence of trade and we hope this project will help us explore more of these Big changes generally take place at key pivotal dates due to shifts in economic, political and environmental factors and historians will look for these peaks and sharp changes. Changes such a huge increase in demand for rubber because of the bicycle craze!
  6. Lost in translation? With academic historians, informatics researchers, visualisation experts, specialists in geospatially enabled databases and a social media specialist gathered together in one small room with a lot of coffee we knew we’d have to do a lot of talking to explain our very different positions. For a start our informatics researchers are used to beginning with a hypothesis whilst our historical researchers are much more likely to take a grounded research approach. This is a really different way to plan and conduct work and we need to understand where we’re all coming from. The tools this project creates need to enable historians in their processes and we must be careful to build something that meets specific needs and appropriate expectations. At the same time, as a project team, we also need to be working together to ensure our publications schedules make sense so we needed to spend some time getting up to speed on which conferences matter in each discipline, where we can work collaboratively on papers and publications, and what types of research outputs are most important for the project partners.
  7. Image of tape storage.

    The History of Tape Storage by Flickr user Pargon

    Storage solutions: a database is not just “a database”, just like furniture from a certain Swedish home furnishing chain you need to know the measurements, the aesthetic needs, the future extensibility before you buy. And just like a house you need the right foundations to build something stable, fit for purpose and ready to use. What questions we will be asking of our data are the essential starting point here (see also Aaron’s blog, “The question is key in Trading Consequences” ) – knowing these and some sort of suitable ontology early on helps us ensure we can design the right structure for our database.
  8. History in a changeable climate – part of the the Trading Consequences project is to consider the impact, the consequences, of historical trades. That means looking at different resources and seeing what the most likely environmental impacts of timber trade, cattle trade and so on might be. That means users may want to query our data based on those impact – looking up the kind of trades that might contribute to flooding, that may be reflected in famine, that might be affected by draught, etc. That requires a whole separate ontology for environmental impact that can somehow account for these very interconnected factors – and that is a lot harder than it looks!
  9. Image of a lab

    Harvey W. Wiley conducting experiments in his laboratory by DC Public Library Commons | DCPL Commons on Flickr Commons (click for more information)

    Shipping drugs – no, not a sinister diversification for the project but a reflection of the complexity of trading data. We can look for records of trading particular types of medicines and drugs but sometimes that’s not the right data to look at. Botanical trades also reflects the trading of drugs as some plant material was shipped for later use or processing into pharmaceuticals (for an idea of the type of plants involved take a look at the Alnwick Poison Garden). The same issue applies to leather goods for instance – you might trade the hides, specific goods like leather gloves, perhaps even the whole cow. All of those trades may reflect leather trade but understanding, combining and querying that data poses some challenges.
  10. Pithy headings! They matter! Part of our project meeting was considering how we communicate the project. As well as learning to use pithy headings, images, bullet points and other web-friendly formatting, we also found out that blog posts should usually be no more than 200-300 words. We also discussed how people access this site on other devices, particularly mobiles. Although we are working on historical data a lot of us are using smart phones and they have smaller screens and differing requirements. We agreed to apply a new mobile theme – so do try reading this blog on your phone and let us know if you like it!

We hope that gave you a flavour of our kick off meeting. It took place over two days so we’ve obviously trimmed it down a lot but if you have any questions, comments or suggestions do add it here and we’ll get back to you.

Trading Consequences at the Geospatial in the Cultural Heritage Domain Event

Earlier this month Claire Grover, one of the Trading Consequences team based at University of Edinburgh Schools of Informatics, gave a presentation on the project at the JISC GECO Geospatial in the Cultural Heritage Domain event in London.

The presentation gives a broad overview of the Trading Consequences project and the initial text mining work that is currently taking place. The slides are now up on SlideShare and the audio recording of Claire’s talk will also be available here shortly:



You can also read a liveblog of all of the talks, including Claire’s, over on the JISC GECO blog.