Easily update R packages after a fresh install

A new version of R was released at the end of October, leading to the persistent issue of how to shift over to the new version with minimal disruption to your workflow.

The most troublesome part of this shift for me is transitioning my packages to the new version of R. There are plenty of ways to do this, many of which are included in this StackOverflow post.

  • You can manually copy the “library” folder from an older R directory to the newer one, then update packages via R

  • You can install the “installr” package

  • You can avoid the problem entirely by setting .libPaths() to another directory on your computer

All good options. But I thought I’d share a few lines of code that do the work for me. The approach is to (1) identify the library for the previous version of R, (2) get a list of the installed packages based on subdirectory names (excluding those associated with the basic functioning of R), and (3) install them to your new parent folder.

Here’s the code:

### step 1: find library folder in previous version of R
dirs <- dir(mainDir <- strsplit(.libPaths()[1], split = "R-")[[1]][1])

### or the target folder can be manually entered
# mainDir <- "C:/Program Files/R/R-3.3.0/library/"
# dirs <- dir(mainDir)

### step 2: get a vector with names of installed, non-base packages
pkgs <- dir(path = mainDir)
pkgs.new <- pkgs[!pkgs %in% dir(mainDir)]

### step 3: re-install!



Non-point source pollution: Using expand.grid for sensitivity analysis

Sometimes things are defined by what they aren’t.”Invertebrates” is a perfect example; despite describing more than 97% of all animal species, invertebrates are defined relative to the 3% of species that have a particular sequence of bones.

“Non-point sources” of nutrients offer a similar example. Non-point sources are diffused over the landscape (agricultural runoff, septic tanks) rather than concentrated inputs from a single point, such as a wastewater treatment plant. In many coastal settings, non-point sources presently contribute much more than point sources to coastal nutrient loads.

Given their importance to coastal nutrient budgets, the question of how to address non-point sources is an urgent one for the coastal science and policy communities. Where septic tanks dominate non-point source nutrient budgets, retrofitting septic tanks is one option that deserves consideration.

This is an interesting problem to think about because there are spatial and temporal dimensions. Nutrients in septic waste enter coastal waters by traveling with groundwater. The travel time between a septic tank and coastal waters can be simplistically thought of as increasing with distance from the coast (Fig. 1).



Figure 1. A coastal watershed with two septic tanks, each having different travel times to coastal waters. DIN = dissolved inorganic nitrogen


The goal of an alternative septic system design is to take some of the nitrogen that would end up in coastal waters and denitrify it on-site, sending it to the atmosphere as inert N2 or as the less-benign N2O (Fig. 2).



Figure 2. A coastal watershed with septic systems that intercept nitrogen on-site. Less dissolved inorganic nitrogen (DIN) is sent to the estuary in this case, and more nitrogen is sent to the atmosphere as N2 and N2O.


Read more of this post

Marsh loss: Submergence vs. herbivory

Estuaries and Coasts recently accepted a really cool paper that I co-authored with Rebecca Schultz and Shimi Anisfeld. The paper (available on ResearchGate) describes an effort to document and understand the fate of two Long Island Sound marshes that are both experiencing vegetation loss. At both marshes, the areas we were working in saw ~18% vegetation loss between 1974-2012.

But that’s where the similarities mostly end. At one marsh, Jarvis Creek in Branford, CT, the vegetation loss was primarily due to purple marsh crab (Sesarma reticulatum) herbivory (Fig. 1).



Figure 1. A purple marsh crab guarding its quarry (photo: Rebecca Schultz).


At the second marsh, Sherwood Island State Park in Westport, CT, the vegetation losses were largely due to a disparity between sea level rise and surface accretion that had been accumulating over several decades.


Read more of this post

Hearing on federal response to Flint’s water crisis

Tomorrow morning (3 February) at 9a the House Oversight Committee will hold a hearing on the federal response to the Flint, MI water crisis. The hearing can be streamed live here: https://oversight.house.gov/hearing/examining-federal-administration-of-the-safe-drinking-water-act-in-flint-michigan/



Read more of this post

Elevation and sediment dynamics: A diversity of tactics

A long-term research project I’ve been involved with has just been published by Estuarine, Coastal and Shelf Science. The paper is available for free at this link until March 13th 2016. The paper describes our understanding of the trajectories of two salt marshes that are following disparate paths; one that’s deteriorating and one that’s stable (Fig. 1).

Microsoft Word - manuscript

Figure 1. Sherwood (A) and Jarvis creek (B) marshes. See our manuscript for a detailed explanation (link in text above).

Read more of this post

Loading NCDC data into R


The National Climatic Data Center (NCDC) is a great source of meteorological and climate data in the US. There’s a huge need for the datasets like their Local Climatological Data reports: long-term, high-resolution data gathered simultaneously at many locations.

I think overall their data are well archived and consistently reported. However, there are a couple shortcomings that limit the use of their data.

First, the website that mediates access to LCD reports makes it time-consuming to download them. This is especially a problem if you want monthly reports from multiple years or stations, since there’s apparently no way to download more than a single station-month of data at a time.

A second obstacle to usage is that their LCD reports aren’t formatted to be machine readable. Annual reports (and monthly reports older than ~15 years) can only be downloaded as pdfs, requiring an additional step of pdf parsing before the text can be worked with. (Changes caused during parsing derailed an earlier attempt I made at digitizing the annual summary LCDs.) Monthly reports usually have the additional option of comma delimited text files, but even these are organized sub-optimally.

It seems genuinely challenging to offer data that is useful for analysts (whose methods and demands are constantly changing), while also satisfying the need to present data consistently from year to year. On the whole, I think NCDC does a good job of reconciling these potentially competing goals.

After admiring NCDC’s data for a while, I finally had occasion to write some R code that reads monthly Local Climatological Data reports and extracts some simple summary data. I think this code could easily be expanded to report any of the data in the LCDs.


Read more of this post

Building a conductivity probe, part 2

In my last post I described code that reads data from a Decagon GS3 soil probe to the Arduino serial monitor. Since then, I’ve added an LCD screen and put the probe in some housing, creating a low-cost, research-quality field instrument. Below are the equipment and code I used.




The probe is unavoidably expensive, but by using $30 in hardware we can avoid paying $500 for a proprietary handheld unit to communicate with the probe. And, we have more control over how data are calculated and reported.




The connections below are slightly different from my last post. I didn’t want to lose an Arduino Uno to this project, so I switched to using a cheaper, smaller Trinket. The pinout is a little different on the Trinket, so I had to shift a few digital pins around. Figure 1 shows the schematic:

Figure 1. Schematic showing hardware connections

Figure 1. Schematic showing hardware connections


Read more of this post