Author: Roger Bivand, Edzer Pebesma

# sp evolution status: examples of migration from retiring packages

#### Roger Bivand, Edzer Pebesma

TOC

[DOWNLOADHERE]

Summary:

This is the fourth report on the R-spatial evolution project and is addressed to maintainers of packages and workflows using `sp` classes, methods and functions. The project involves the retirement (archiving) of `rgdal`, `rgeos` and `maptools` during 2023. The first report set out the main goals of the project. The second report covered progress so far, steps already taken, and those remaining to be accomplished. The third gave detailed guidance for maintainers of packages using the retiring packages.

From June 2023 (`sp` version 2.0-0), the internal evolution status setting of `sp` will be changed from “business as usual” to “use `sf` instead of `rgdal` and `rgeos`”. Packages depending on `sp` may need to add `sf` to their weak dependencies, and to monitor any changes in output. From this version 2.1-0, use of `rgdal` has been dropped, so setting the internal evolution status has been removed.

The final step occurred during October 2023, `rgdal`, `rgeos` and `maptools` were archived on CRAN, and packages with strong dependencies on the retiring packages must be either upgraded to use `sf`, `terra` or other alternatives or work-arounds by or before that time.

This spreadsheet lists methods and functions in retiring packages and `sp` found by `pkgapi`. They are listed by function name as used by packages, and the analogous list by package may be found of functions is here.

# `sp` evolution status

Repeating from the second and third blogs:

As mentioned in our first report, `sp` on CRAN has been provided with conditional code that prevents `sp` calling most code in `rgdal` or `rgeos`. This can be enabled before loading `sp` by setting e.g.:

    options("sp_evolution_status"=2)
    library(sp)

for checking packages under status

- `0`: business as usual,
- `1`: stop if `rgdal` or `rgeos` are absent, or
- `2`: use `sf` instead of `rgdal` (currently no use of `rgeos` is considered)

or alternatively can be set as an environment variable read when `sp` is loaded, e.g. when running checks from the command line by

    _SP_EVOLUTION_STATUS_=2 R CMD check

This construction should permit maintainers to detect potential problems in code. `devtools::check()` provides the `env_vars=` argument, which may be used for the same purpose.

From `sp 1.6.0` published on CRAN 2023-01-19, these status settings may also be changed when `sp` is loaded, using `sp::get_evolution_status()` returning the current value, and `sp::set_evolution_status(value)`, where value can take the integer values `0L`, `1L` and `2L`.

## Use of `sp` affected functions and methods

Under evolution status `2`, which will become the default in June 2023, no use is made of `rgdal` in `sp::CRS`, `sp::is.projected`, and `sp::spTransform`, rather using `sf` to access `PROJ` through `GDAL`. A warning is given if `sf` is not available. This is similar to behaviour under legacy `sp` and evolution status `0` with regard to the use of `rgdal` to access `PROJ`.

`sp::CRS` is used by the following packages (`pkgapi` May 2, 2023):

    adehabitatHR adehabitatHS adehabitatLT adehabitatMA AFM AGPRIS angstroms
    animaltracker anipaths antaresViz aqp AquaBEHER atakrig ausplotsR

    bamlss bfsMaps biogeo bioRad biosurvey birdring birdscanR bivariatemaps
    bRacatus briskaR

    canadianmaps changeRangeR chronosphere cleangeo cmsafops cmsafvis
    ConR CoordinateCleaner cruts ctmm

    dartR DEPONS2R dggridR dismo dispeRse DRHotNet dtwSat dynamAedes dynamicSDM

    ecospat elevatr EMbC epitweetr

    fasterize FIESTAutils FLightR FRK

    GapAnalysis gDefrag gdistance GeNetIt GeoAdjust GeodesiCL geodiv geofacet
    geogrid geojsonio geomerge geoviz gfcanalysis ggOceanMaps gmGeostats
    googletraffic GPSeqClus grainscape graticule gwer gwfa

    hero Hmsc hydroTSM

    iccTraj inlabru INLAspacetime intamap intSDM IsoriX itcSegment

    kehra

    lakemorpho leaflet letsR lgcp loa

    macroBiome mapedit MapGAM mapmisc mapview MEDITS meteo meteoForecast
    meteoland micromapST MinBAR momentuHMM morphomap move

    nodiv

    oceanic OpenStreetMap OSMscale

    phyloregion places plotKML prevR PWFSLSmoke

    QRAGadget quadmesh quickmapr quickPlot

    RAC raster rasterVis rbgm rcanvec rCAT RCGLS RchivalTag rcrimeanalysis
    ReadDIM red RgoogleMaps riverdist rleafmap rosm rpostgis RSIP rtop RWmisc

    sdm sdStaf SeerMapper seg shadow SimSurvey soilassessment SongEvo spacetime
    SpaDES.tools SpatialEpi spatialfusion SpatialGraph spatsoc spatsurv spbabel
    spdep sperich spex spgwr stppSim stxplore SUNGEO SurfaceTortoise SWTools

    tiler track2KBA TrajDataMining trajectories trip

    uavRmp

    vec2dtransf

    waver WEGE wildlifeDI wkb wux

    zoon

`sp::is.projected` is used by:

    GapAnalysis GeNetIt intkrige ptools riverdist surveillance track2KBA trip

and `sp::spTransform` by:

    AGPRIS antaresViz

    biosurvey bivariatemaps

    canadianmaps chronosphere CoordinateCleaner ctmm

    elevatr epitweetr

    fastmaRching FLightR

    GeoAdjust GeodesiCL geoviz ggOceanMaps

    icosa inlabru INLAspacetime

    loa

    macroBiome mapedit mapview micromapST MinBAR momentuHMM mregions

    quickmapr

    raster rcanvec rcrimeanalysis reproducible RgoogleMaps riverdist rosm rpostgis

    SeerMapper shadow SUNGEO

    track2KBA

    uavRmp

Reverse dependency checks of May 4, 2023 showed only 2 packages failing `CMD check` under `_SP_EVOLUTION_STATUS_=2`, and their difficulties are mentioned below (github issues have been raised offering maintainers mitigations). The salient reasons are given below so that others facing similar apparent intractible problems in workflows can perhaps benefit. Since other packages passed `CMD check`, there are no strong reasons not to flip `sp`’s evolution status in June 2023 from `rgdal` to `sf`.

## Possible difficulties

One possible source of difference is that the strings (WKT2_2019, Proj4) returned from `rgdal` came straight from `PROJ`, but `sf` accesses `PROJ` through `GDAL`.

Another source of difference is that `rgdal` used `sp` S4 classes with inheritance, but `sf` uses S3 classes which do not (seem to) treat inheritance in the same way. So workflows using `sp` and classes, so coercion to `sp` classes before `sp::spTransform` is required, and reconstruction of the classes extending `sp` classes after return is probably required.

A further potential cause of confusion is that round-tripping by coercion to `sf` perhaps applying an operation, and returning to `sp` may break rarely used `sp` methods, such as:

    library(sp)
    getMethod("$", "SpatialPoints")

    Method Definition:

    function (x, name)
    {
        if (name %in% coordnames(x))
            return(x@coords[, name])
        if (!("data" %in% slotNames(x)))
            stop("no $ method for object without attributes")
        x@data[[name]]
    }
    <bytecode: 0x5a1f44fd0258>
    <environment: namespace:sp>

    Signatures:
            x
    target  "SpatialPoints"
    defined "SpatialPoints"

which accesses columns by name no longer in the `data.frame` in the data slot through the column names of the matrix of points:

    data(meuse)
    names(meuse)

     [1] "x"       "y"       "cadmium" "copper"  "lead"    "zinc"    "elev"
     [8] "dist"    "om"      "ffreq"   "soil"    "lime"    "landuse" "dist.m"

    coordinates(meuse) <- ~ x + y
    names(meuse)

     [1] "cadmium" "copper"  "lead"    "zinc"    "elev"    "dist"    "om"
     [8] "ffreq"   "soil"    "lime"    "landuse" "dist.m"

    colnames(slot(meuse, "coords"))

    [1] "x" "y"

    str(meuse$x)

     Named num [1:155] 181072 181025 181165 181298 181307 ...
     - attr(*, "names")= chr [1:155] "1" "2" "3" "4" ...

So far so good, but round-tripping if `sf` is available changes the column names of the matrix of points:

    run <- requireNamespace("sf", quietly = TRUE)

    meuse_rt <- as(sf::st_as_sf(meuse), "Spatial")
    names(meuse_rt)

     [1] "cadmium" "copper"  "lead"    "zinc"    "elev"    "dist"    "om"
     [8] "ffreq"   "soil"    "lime"    "landuse" "dist.m"

    colnames(slot(meuse_rt, "coords"))

    [1] "coords.x1" "coords.x2"

Consequently, code using this legacy short cut may be a source of difficulty.

## Why not `terra`?

The reason for choosing to replace `rgdal` by `sf` for these three methods has been the absence of an independent object for coordinate reference systems matching `"CRS"` in `sp` and `"crs"` in `sf`.

It would be possible to create an empty `"SpatVector"` object and assign a coordinate reference system, coerce to `sp`, and extract the `"CRS"` object:

    v <- terra::vect(matrix(c(0, 0), nrow=1), crs="OGC:CRS84")

    library(raster)
    slot(as(v, "Spatial"), "proj4string")

    Coordinate Reference System:
    Deprecated Proj.4 representation: +proj=longlat +datum=WGS84 +no_defs
    WKT2 2019 representation:
    GEOGCRS["unknown",
        DATUM["World Geodetic System 1984",
            ELLIPSOID["WGS 84",6378137,298.257223563,
                LENGTHUNIT["metre",1]],
            ID["EPSG",6326]],
        PRIMEM["Greenwich",0,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8901]],
        CS[ellipsoidal,2],
            AXIS["longitude",east,
                ORDER[1],
                ANGLEUNIT["degree",0.0174532925199433,
                    ID["EPSG",9122]]],
            AXIS["latitude",north,
                ORDER[2],
                ANGLEUNIT["degree",0.0174532925199433,
                    ID["EPSG",9122]]]]

but this seems somewhat forced. If there is substantial demand for an alternative mechanism using `terra` and `raster` in place of `sf`, then a well-tested pull request would be considered. It is understandable that Windows or macOS users who otherwise would not need to install `sf`, and who are already using `terra` might prefer to stay with `terra` inside `sp`, but input to the evolution project would be needed to take any action. It may rather be simpler for users with such workflows to migrate to `terra` instead of sustaining a mixed `sp`, `raster` vector combination when `terra` can probably cover all needs.