Summary

• Make a quarto book
• Add an appropriate license CC BY-NC-4.0
  • Can share/adapt, must attribute
  • Cannot use materials commercially
• Add a README
• Put it on github
• Have it render as an online book when you make changes

Follow along with github repo: “course-book-template”

I have made a repo on github “course-book-template” that details each of these steps.

Make a quarto book

The quarto docs on starting a book are excellent so I would recommend starting there: https://quarto.org/docs/books/#quick-start.

But essentially, you run:

quarto create project book .

It will then guide you through creating a title, etc

commit of adding the book

Add an appropriate license

It is important to pick a license early. This helps protect your work, and also makes it clear to others how to reference and use your work. Personally, I like CC BY-NC-4.0, this gives you these conditions:

• Can share/adapt, must attribute
• Cannot use materials commercially

Note that this is different to the very common CC-BY, which does allow commercial usage.

If you aren’t sure about licenses for your purpose, it would be worthwhile checking out the chooser here https://creativecommons.org/chooser/

In using this, I discovered another useful license, CC-BY-NC-SA 4.0 Which builds off of CC-BY-NC, but with one additional clause:

ShareAlike - If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original.

This is sometimes known as “copyleft”. This is sometimes seen as too restrictive. Consult with your community about what the standards are. Another useful place to read up on licenses is the “licensing” chapter in the R packages book by Hadley Wickham and Jenny Bryan

I add a LICENSE file, and also a license.qmd chapter, as well as add the license to the README.

commit of adding the license

Using a README

I think it is worthwhile adding a few key sections to your README file:

• Details: like the abstract - the hook!
• Prerequisites: What do you expect learners to know?
• Learning outcomes: What will they walk away knowing?
• Schedule: An outline of each hour of learning, optionally with a timetable

commit of adding the README

Put it on online - github

Your course should live somewhere public! You can see for example our course materials here: https://github.com/njtierney/course-book-template

Have it render when you make changes

You can use github actions to render your book. This is really neat, and means your book will be rendered anytime you push changes. It means you don’t need to push HTML, just the quarto files.

Rhere are a few different ways you can manage this, I happen to like using github pages.

There are some really nice instructions on the quarto website on how to set up github pages - https://quarto.org/docs/publishing/github-pages.html#github-action

However, I have found a slightly different setup, which I will share here.

This involves using a DESCRIPTION file to track the R packages that you use. The reason we need to do this is to make sure when we render our book, that all the R packages we need are installed. There are probably other ways around this, and I’d love to hear them, but this is what I have found works.

Here is the first step where I add a dependency, in this case, tidyverse.

commit of adding this tidyverse code

Then add the DESCRIPTION file with:

usethis::use_description(check_name = FALSE)

I then edited mine to look like this:

Package: course-book-template
Title: A book about some things
Version: 0.0.0.9000
Authors@R: 
  c(
  person(
    given = "Nicholas",
    family = "Tierney",
    email = "nicholas.tierney@gmail.com",
    role = c("aut", "cre"),
    comment = c(ORCID = "https://orcid.org/0000-0003-1460-8722")
    )
  )
Description: Course materials for your topic. This should have two sentences.
License: CC-BY-NC 4.0 + file LICENSE
Encoding: UTF-8
Language: en-GB
Roxygen: list(markdown = TRUE)
RoxygenNote: 8.0.0

commit

You can then add your package dependency into Imports or Depends. Which one you use is normally very important for R package development, but the reason we are using a DESCRIPTION file here is to track our dependencies.

usethis::use_package("tidyverse", type = "Depends")

commit

I then add the github actions - you can actually just refer to a file, so this will work:

use_github_action(url = "https://github.com/njtierney/gentlegit/blob/main/.github/workflows/quarto-publish.yml")

This will give you a message like the following:

✔ Creating .github/.
✔ Adding "^\\.github$" to .Rbuildignore.
✔ Adding "*.html" to .github/.gitignore.
✔ Creating .github/workflows/.
✔ Saving
  "njtierney/gentlegit/.github/workflows/quarto-publish.yml@main"
  to .github/workflows/quarto-publish.yml.

commit

Also, probably a good time to add a .gitignore file. This is a good idea to make sure you don’t commit HTML files (they can be really large,a nd we don’t need them), or other file types that might be really large, or have sensitive information in them.

usethis::use_git_ignore("*.pdf")

Will create the file, and tell git to never commit a PDF.

I edit my .gitignore file to look like the following:

/.quarto/
**/*.quarto_ipynb
.Rproj.user
.Rhistory
.RData
.Ruserdata
dev
docs
/.quarto/
*.aux
*.log
*.pdf
*.tex
*.toc
*.rds
*_files
*_cache
*.html
.DS_Store

commit

Once all this is said and done, you will still need to run some commands in your terminal:

quarto publish gh-pages

This should then produce a question like:

nick course-book-template[main] > quarto publish gh-pages
? Publish site to https://njtierney.github.io/course-book-template/ using gh-pages? (Y/n) › 

reply “Y”

You will then get some code that looks like:

Switched to a new branch 'gh-pages'
[gh-pages (root-commit) 8ccc5e0] Initializing gh-pages branch
remote: 
remote: Create a pull request for 'gh-pages' on GitHub by visiting:        
remote:      https://github.com/njtierney/course-book-template/pull/new/gh-pages        
remote: 
To https://github.com/njtierney/course-book-template.git
 * [new branch]      HEAD -> gh-pages
Switched to branch 'main'
Your branch is up to date with 'origin/main'.
From https://github.com/njtierney/course-book-template
 * branch            gh-pages   -> FETCH_HEAD

And then some rendering code that will look like:

Rendering for publish:

[1/4] index.qmd
[2/4] intro.qmd


processing file: intro.qmd
1/3                  
2/3 [unnamed-chunk-1]
3/3                  
output file: intro.knit.md

...

(|) Deploying gh-pages branch to website (this may take a few minutes)

Wait a few minutes, as it asks you.

Then you should see something like:

[✓] Deploying gh-pages branch to website (this may take a few minutes)
[✓] Published to https://njtierney.github.io/course-book-template/

NOTE: GitHub Pages deployments normally take a few minutes (your site updates
will be visible once the deploy completes)

Your website probably won’t be visible just yet, which feels a touch annoying, but you can keep an eye on it on the “actions” tab, e.g., https://github.com/njtierney/course-book-template/actions

Once this has lit green (hopefully it has!)

You should go to your “about” section, and click on the setting cog:

Then tick the box that says “Use your GitHub Pages website”

This adds your GitHub Pages website onto the repo, and it looks pretty neat.

There are more things you can do, like configuring your own custom website instead of using github.

So, instead of https://njtierney.github.io/course-book-template/, you could have: “course-book-template.com”.

And that’s it!

Installation is now faster

install.packages() and download.packages() download packages simultaneously using libcurl, significantly reducing download times when installing or downloading multiple packages.

This is cool! Installing packages is now much faster! I love this. Generally, I use pak, which I believe already does this. But it’s a nice win for vanilla R to know I can download things faster.

Data

There are two new datasets added to R! The last time (that I could find) a new dataset was added to R was in R version 3.0.2, which, going to the release dates, was in September 2013. So, nearly 12 years ago! That NEWS item read:

Dataset npk has been copied from MASS to allow more tests to be run without recommended packages being installed.

You might be familiar with one of these new datasets, the penguins data (!) from the palmerpenguins R package.

artwork by Allison Horst

The NEWS item reads:

New datasets penguins and penguins_raw thanks to Ella Kaye, Heather Turner, and Kristen Gorman.

This is huge! Well, I think it is. Now we don’t need to write library(palmerpenguins) if you want to use the penguins or penguins_raw data. Looking at the examples, we can see the following base visualisations:

## bill depth vs. length by species (color) and sex (symbol):
## positive correlations for all species, males tend to have bigger bills
head(penguins)
#>   species    island bill_len bill_dep flipper_len body_mass    sex year
#> 1  Adelie Torgersen     39.1     18.7         181      3750   male 2007
#> 2  Adelie Torgersen     39.5     17.4         186      3800 female 2007
#> 3  Adelie Torgersen     40.3     18.0         195      3250 female 2007
#> 4  Adelie Torgersen       NA       NA          NA        NA   <NA> 2007
#> 5  Adelie Torgersen     36.7     19.3         193      3450 female 2007
#> 6  Adelie Torgersen     39.3     20.6         190      3650   male 2007
names(penguins)
#> [1] "species"     "island"      "bill_len"    "bill_dep"    "flipper_len"
#> [6] "body_mass"   "sex"         "year"
sym <- c(1, 16)
pal <- c(
  "darkorange",
  "purple",
  "cyan4"
  )
plot(
  bill_dep ~ bill_len, 
  data = penguins, 
  pch = sym[sex], 
  col = pal[species]
  )

It is worth noting that the variable names are slightly different in the new base R version - bill_len (not bill_length) bill_dep (not bill_depth), and flipper_len (not flipper_length), which they note in the helpfile: “The penguins data has some shorter variable names than the palmerpenguins version, for compact code and data display.”

There has been a bit of work done on the penguins data, and I wanted to share the relevant references:

• Gorman, K. B., Williams, T. D. and Fraser, W. R. (2014) Ecological Sexual Dimorphism and Environmental Variability within a Community of Antarctic Penguins (Genus Pygoscelis). PLoS ONE 9, 3, e90081; doi:10.1371/journal.pone.0090081.

• Horst, A. M., Hill, A. P. and Gorman, K. B. (2022) Palmer Archipelago Penguins Data in the palmerpenguins R Package - An Alternative to Anderson’s Irises. R Journal 14, 1; doi:10.32614/RJ-2022-020.

• Kaye, E., Turner, H., Gorman, K. B., Horst, A. M. and Hill, A. P. (2025) Preparing the Palmer Penguins Data for the datasets Package in R. doi:10.5281/zenodo.14902740.

And there’s ANOTHER dataset!

New dataset gait thanks to Heather Turner and Ella Kaye, used in examples.

I hadn’t come across this before, the helpfile (?gait) reads:

Hip and knee angle (in degrees) through a 20 point movement cycle for 39 boys.

Which is stored as a 3D array, as stated in the helpfile:

A 3-dimensional array with dimensions c(20, 39, 2) giving the “Hip Angle” and “Knee Angle” (in degrees) for 39 repetitions of a 20 point gait cycle (over standardized gait times).

head(gait)
#> , , Variable = Hip Angle
#> 
#>        Subject
#> Time    boy1 boy2 boy3 boy4 boy5 boy6 boy7 boy8 boy9 boy10 boy11 boy12 boy13
#>   0.025   37   47   46   37   20   57   46   46   46    35    38    35    34
#>   0.075   36   46   44   36   18   48   38   46   42    34    37    31    31
#>   0.125   33   42   39   27   11   44   33   43   37    29    33    29    27
#>   0.175   29   34   34   20    8   35   25   40   34    28    29    26    23
#>   0.225   23   27   33   15    7   31   18   36   31    19    26    22    19
#>   0.275   18   21   27   15    5   27   15   30   25    15    20    19    15
#>        Subject
#> Time    boy14 boy15 boy16 boy17 boy18 boy19 boy20 boy21 boy22 boy23 boy24 boy25
#>   0.025    43    43    40    51    52    36    35    46    43    55    39    37
#>   0.075    41    37    41    49    46    33    37    38    41    51    38    34
#>   0.125    36    35    36    45    41    28    33    30    37    47    31    30
#>   0.175    31    28    32    39    35    22    27    23    30    41    27    27
#>   0.225    26    26    27    31    31    18    22    17    24    35    21    26
#>   0.275    20    21    20    23    24    13    14    13    16    30    14    19
#>        Subject
#> Time    boy26 boy27 boy28 boy29 boy30 boy31 boy32 boy33 boy34 boy35 boy36 boy37
#>   0.025    36    36    42    38    46    54    52    32    46    46    48    44
#>   0.075    33    33    40    34    47    48    44    28    41    44    42    41
#>   0.125    28    30    40    30    44    44    44    26    38    40    42    38
#>   0.175    22    28    34    23    37    37    33    22    31    35    35    32
#>   0.225    18    21    23    17    29    30    28    19    25    31    30    24
#>   0.275    13    15    15    12    23    27    27    13    20    25    23    18
#>        Subject
#> Time    boy38 boy39
#>   0.025    55    48
#>   0.075    56    50
#>   0.125    51    47
#>   0.175    46    42
#>   0.225    41    37
#>   0.275    36    29
#> 
#> , , Variable = Knee Angle
#> 
#>        Subject
#> Time    boy1 boy2 boy3 boy4 boy5 boy6 boy7 boy8 boy9 boy10 boy11 boy12 boy13
#>   0.025   10   16   18    5    2   15   13   14   15     9    13     7     9
#>   0.075   15   25   27   14    6   17   16   17   20    22    24     8    14
#>   0.125   18   28   32   16    6   23   22   18   23    25    27    11    16
#>   0.175   18   25   32   17    6   23   17   19   26    21    23    12    15
#>   0.225   15   18   28   10    5   20   12   19   25    10    18     8    15
#>   0.275   14   12   23    8    6   19    9   15   21     9    13     6    12
#>        Subject
#> Time    boy14 boy15 boy16 boy17 boy18 boy19 boy20 boy21 boy22 boy23 boy24 boy25
#>   0.025    15     6    11    24    16    16     7    21    11    12     8    11
#>   0.075    20    11    19    32    20    20    13    24    14    17    12    20
#>   0.125    22    20    30    35    21    22    14    25    14    20    14    22
#>   0.175    22    18    28    33    20    21    17    21    11    20    13    21
#>   0.225    21    13    25    29    18    20    14    16     8    18    12    21
#>   0.275    19     9    17    24    14    20     8     9     5    12     9    17
#>        Subject
#> Time    boy26 boy27 boy28 boy29 boy30 boy31 boy32 boy33 boy34 boy35 boy36 boy37
#>   0.025    16    19    13    11    17    20    18     9     8     9    13    19
#>   0.075    20    26    23    15    25    20    18    12    10    18    18    23
#>   0.125    22    28    30    19    30    22    25    16    17    19    27    26
#>   0.175    21    28    28    20    30    16    23    15    16    19    26    25
#>   0.225    20    24    19    18    27    10    18    14    12    19    25    21
#>   0.275    20    18    10    17    22    10    19    11    10    15    18    18
#>        Subject
#> Time    boy38 boy39
#>   0.025    16    14
#>   0.075    23    25
#>   0.125    28    32
#>   0.175    28    34
#>   0.225    25    30
#>   0.275    21    20

With the examples having a nifty data visualisation:

plot(
  gait[, 1, ], 
  type = "b",
  xlim = range(gait[,,1]), 
  ylim = range(gait[,,2]), 
  xlab = "Hip Angle", 
  ylab = "Knee Angle", 
  main = "'gait' data : Boy 1"
)
mtext(
  "all other boys", 
  col = "thistle"
  )
grid()
matlines(
  gait[, -1, 1], 
  gait[, -1, 2], 
  type = "l", 
  lty = 1, 
  col = adjustcolor("thistle", 1/3)
  )

It is worth noting the references for this data:

• Olshen, R. A., Biden, E. N., Wyatt, M. P., and Sutherland, D. (1989) Gait Analysis and the Bootstrap. Annals of Statistics 17, 4, 1419–1440. doi:10.1214/AOS/1176347372

• Ramsay, J. O., and Silverman, B. W. (2006) Functional Data Analysis, 2nd ed., New York: Springer.

• Ramsay, J. (2023) fda: Functional Data Analysis. R package version 6.1.4, https://CRAN.R-project.org/package=fda.

BLAS and LAPACK update

The bundled BLAS and LAPACK sources have been updated to those shipped as part of January 2025’s LAPACK 3.12.1.

These help power many linear algebra operations in R, and there will likely be speed improvements and other efficiencies. So, worth installing for that alone.

A note on reading NEWS files

I have tried to read the R NEWS files since about 2017 or 2018. I believe I heard Roger Peng say he read the R NEWS files on the Not So Standard Deviations podcast with Hilary Parker. It seemed like a good idea, so I have tried to do the same.

It usually involves me reading over the NEWS file and looking out for things that seem relevant to me and my work. I don’t always understand everything, in fact, I would say I don’t understand a lot of it. But every now and again something comes up that is important to me. For example, the NEWS item for R 4.0.0 about matrices and arrays:

matrix objects now also inherit from class “array”, so e.g., class(diag(1)) is c(“matrix”, “array”). This invalidates code incorrectly assuming that class(matrix_obj)) has length one.

S3 methods for class “array” are now dispatched for matrix objects.

Resulted in a breaking change in greta, that I was able to identify and fix.

Another time that I find it useful to know how to look up the NEWS file is remembering when some key new features were introduced. For example, knowing that the native pipe |> was introduced in 4.1.0, but the underscore _ placeholder was only introduced in 4.2.0 is really useful if I’m developing packages to know which version of R to depend on.

I also often visit the “Previous releases of R for Windows” page to see when a version of R was released. This is to check if a version of R is more than three years old, which to me, is about old enough that people shouldn’t be depending on it on a regular basis, or if they are and it is mission critical, they would also have other things set up to maintain it. The tidyverse maintains backward compatibility with the past 5 (minor) versions of R - which ends up being about 4 years. I figure 3 years should be good enough for my work.

Yes, it is annoying to reinstall R packages again.

When you install a new minor version of R (that is, when you change its minor version, for example going from 4.1.0 to 4.2.0, not 4.1.0 to 4.1.1), you need to install all your R packages again.

Sure, it’s annoying. But I see it as an opportunity to start afresh. I usually do something like the following

install.packages("pak")
library(pak)
pkg_install("tidyverse", dependencies = TRUE)
pkg_install("devtools", dependencies = TRUE)
pkg_install("naniar", dependencies = TRUE)
pkg_install("blogdown", dependencies = TRUE)
pkg_install("greta", dependencies = TRUE)
pkg_install("brolgar", dependencies = TRUE)
pkg_install("targets", dependencies = TRUE)
pkg_install("ropensci/geotargets", dependencies = TRUE)
pkg_install("milesmcbain/datapasta", dependencies = TRUE)

And this gets me 90% of the way there.

It is worth noting that if you’re a windows user, you can use the installr package to update R and other dependencies.

And on that note it’s ALSO worth mentioning the rig command line utility. This allows you to more easily install different, and release versions of R. If you’ve seen Bob Rudis’s RSwitch software, this is the updated version of this (Bob has archived RSwitch).

Why should I use geotargets and targets?

You could benefit from using targets and geotargets if you do geospatial data analysis involving rasters or shapefiles, specifically with terra or stars R packages. For example, if you are doing large downloads of rasters, then operations like cropping, reprojection, resampling, and masking.

The main benefit to using targets and geotargets is your analysis will only rerun when you change relevant parts of your data analysis. For example, you might do a lot of geospatial data processing that feeds downstream into a machine learning model to make predictions on bushfire risk. Writing with targets and geotargets means if you change the parts of the code that related to the machine learning components, then only the relevant parts with machine learning would change. This means you can save time by avoiding running computational expensive spatial data processing.

For more details on what targets is, and why we need geotargets, I would recommend reading the 0.1.0 release blog post, as well as reading the {targets} manual. The “Get started in 4 minutes” guide to targets is also excellent.

Main changes in 0.2.0?

In addition to smaller changes and improvements, there are three main additions in this release:

• Support for dynamic branching.
• Preserving spatRaster metadata.
• Support of stars and stars_proxy.

And very in very exciting news, we have a new hex sticker!

Thanks to Hubert Hałun for their work on getting this together, we are really happy with the new sticker!

Dynamic Branching

The main addition in this release is a demonstration of using dynamic branching using a new “target factory” function, tar_terra_tiles(). This allows you to break raster operations into tiles, and then perform these operations on the tiles and combine them together. This means we can break computationally intensive raster operations that work in pixel-wise manner over tiled subsets of the raster. This is useful when, for example, loading an entire raster into memory and doing computations on it results in out of memory errors.

As part of this addition, we created helper functions:

• tile_n()
• tile_grid(), and
• tile_blocksize().

These help us define different extents that we can pass along as different parts of the dynamic branches. You can think of these as tools that we can use to specify how to slice, or tile up, a raster into smaller pieces that we can then do analysis on separately and combine later.

Let’s briefly unpack these, and then show how these would be used in dynamic branching. First let’s read in some example elevation data from terra and plot it:

f <- system.file("ex/elev.tif", package="terra")
r <- rast(f)
plot(r)

tile_n()

We can use tile_n(), which is the simplest of the three. It produces about n tiles in a grid.

r_tile_4 <- tile_n(r, 4)
#> creating 2 * 2 = 4 tile extents
r_tile_4
#> [[1]]
#>      xmin      xmax      ymin      ymax 
#>  5.741667  6.141667 49.816667 50.191667 
#> 
#> [[2]]
#>      xmin      xmax      ymin      ymax 
#>  6.141667  6.533333 49.816667 50.191667 
#> 
#> [[3]]
#>      xmin      xmax      ymin      ymax 
#>  5.741667  6.141667 49.441667 49.816667 
#> 
#> [[4]]
#>      xmin      xmax      ymin      ymax 
#>  6.141667  6.533333 49.441667 49.816667

# some plot helpers
rect_extent <- function(x, ...) {
  rect(x[1], x[3], x[2], x[4], ...)
}
plot_extents <- function(x, ...) {
  invisible(lapply(x, rect_extent, border = "hotpink", lwd = 2))
}

plot(r)
plot_extents(r_tile_4)

plot(r)
tile_n(r, 6) |> plot_extents()
#> creating 2 * 3 = 6 tile extents

tile_grid()

For more control, use tile_grid(), which allows specification of the number of rows and columns to split the raster into. Here we are specify that we want three columns and 1 row:

r_grid_3x1 <- tile_grid(r, ncol = 3, nrow = 1)
r_grid_3x1
#> [[1]]
#>      xmin      xmax      ymin      ymax 
#>  5.741667  6.008333 49.441667 50.191667 
#> 
#> [[2]]
#>      xmin      xmax      ymin      ymax 
#>  6.008333  6.266667 49.441667 50.191667 
#> 
#> [[3]]
#>      xmin      xmax      ymin      ymax 
#>  6.266667  6.533333 49.441667 50.191667
plot(r)
plot_extents(r_grid_3x1)

plot(r)
tile_grid(r, ncol = 2, nrow = 3) |> plot_extents()

tile_blocksize()

The third included helper is tile_blocksize(), which tiles by file block size. The block size is a property of raster files, and is the number of pixels (in the x and y direction) that is read into memory at a time. Tiling by multiples of block size may therefore be more efficient because only one block should need to be loaded to create each tile target. You can find the blocksize with fileBlocksize:

fileBlocksize(r)
#>      rows cols
#> [1,]   43   95

This tells us that it reads in the raster in 43x95 pixel sizes.

The tile_blocksize function is similar to tile_grid, except instead of saying how many rows and columns, we specify in units of blocksize.

If we just run tile_blocksize() on r we get the extents of the specified blocksize:

tile_blocksize(r)
#> [[1]]
#>      xmin      xmax      ymin      ymax 
#>  5.741667  6.533333 49.833333 50.191667 
#> 
#> [[2]]
#>      xmin      xmax      ymin      ymax 
#>  5.741667  6.533333 49.475000 49.833333 
#> 
#> [[3]]
#>      xmin      xmax      ymin      ymax 
#>  5.741667  6.533333 49.441667 49.475000

Which is the same as specifying blocksize for row and column at unit 1:

r_block_size_1x1 <- tile_blocksize(r, n_blocks_row = 1, n_blocks_col = 1)
r_block_size_1x1
#> [[1]]
#>      xmin      xmax      ymin      ymax 
#>  5.741667  6.533333 49.833333 50.191667 
#> 
#> [[2]]
#>      xmin      xmax      ymin      ymax 
#>  5.741667  6.533333 49.475000 49.833333 
#> 
#> [[3]]
#>      xmin      xmax      ymin      ymax 
#>  5.741667  6.533333 49.441667 49.475000
plot(r)
plot_extents(r_block_size_1x1)

Here the block size is the same size for the first two blocks, and then a much more narrow block. This is different to the two other tile methods.

Here the column block size is the full width of the raster.

So we could instead have the blocksize extent be written out to 2 blocks in a row, and 1 block size for the columns:

r_block_size_2x1 <- tile_blocksize(r, n_blocks_row = 2, n_blocks_col = 1)
r_block_size_2x1
#> [[1]]
#>      xmin      xmax      ymin      ymax 
#>  5.741667  6.533333 49.475000 50.191667 
#> 
#> [[2]]
#>      xmin      xmax      ymin      ymax 
#>  5.741667  6.533333 49.441667 49.475000
plot(r)
plot_extents(r_block_size_2x1)

This only works when the SpatRaster points to a file—in-memory rasters have no inherent block size.

sources(r)
#> [1] "/Users/nick/Library/R/arm64/4.4/library/terra/ex/elev.tif"
#force into memory
r2 <- r + 0
sources(r2)
#> [1] ""
#this now errors
tile_blocksize(r2)
#> Error: [aggregate] values in argument 'fact' should be > 0

For more detail on using this in targets, please see the geotargets vignette, “Dynamic branching with raster tiles”

Minor changes in 0.2.0

Other changes include:

• Created utility function set_window() mostly for internal use within tar_terra_tiles().
• Removes the iteration argument from all tar_*() functions. iteration now hard-coded as "list" since it is the only option that works (for now at least).
• Added the description argument to all tar_*() functions which is passed to tar_target().
• Suppressed the warning “[rast] skipped sub-datasets” from tar_terra_sprc(), which is misleading in this context (#92, #104).
• Requires GDAL 3.1 or greater to use “ESRI Shapefile” driver in tar_terra_vect() (#71, #97)
• geotargets now requires targets version 1.8.0 or higher
• terra (>= 1.7.71), withr (>= 3.0.0), and zip are now required dependencies of geotargets (moved from Suggests to Imports)

What’s next?

We have finished developing the main milestones for geotargets, but will continue actively developing it. Soon we will be submitting the package for review by rOpenSci, and subsequently submit the work to the Journal of Open Source Software (JOSS), and then submit to CRAN.

Currently, the next release will focus on adding support for:

• Add support for ArcGIS services using {arcgislayers}

• Adding support for tar_terra_rast on cloud storage

• explore support for {gdalraster}

• Add support for terra SpatVectorProxy, and format=“file” for SpatRaster

You can see the full list of issues for more detail on what we are working on.

Thanks

We would like to thank the R Consortium for generously supporting this project, “{geotargets}: Enabling geospatial workflow management with {targets}”.

We would also like to thank Michael Sumner, Anthony North, and Miles McBain for their helpful discussions throughout, as well as Will Landau for writing targets, and being incredibly responsive and helpful to the issues and questions we have asked as we wrote {geotargets}.

What is targets? Why do I need geotargets?

The targets package is an R package for managing analytic pipelines. It means that you can write out an analysis in a specific manner, and then as you update code, it will only rerun the necessary parts. Essentially it helps you avoid running large pieces of analysis when you don’t need to. To learn more about targets, I’d highly recommend reading the {targets} manual.

Let’s show an example. Let’s say we want to get an example raster file from {terra}, we can do the following:

terra_rast_example <- system.file(
  "ex/elev.tif", 
  package = "terra"
  ) |>
  terra::rast()

terra_rast_example
#> class       : SpatRaster 
#> dimensions  : 90, 95, 1  (nrow, ncol, nlyr)
#> resolution  : 0.008333333, 0.008333333  (x, y)
#> extent      : 5.741667, 6.533333, 49.44167, 50.19167  (xmin, xmax, ymin, ymax)
#> coord. ref. : lon/lat WGS 84 (EPSG:4326) 
#> source      : elev.tif 
#> name        : elevation 
#> min value   :       141 
#> max value   :       547

Here is the equivalent code in a targets pipeline - the reason we want to use {targets} here is we save the results so we don’t need to run them again. In this case the example code doesn’t take long to run. But imagine reading in the raster was hugely time and computer expensive and we didn’t want to do it again. The {targets} package stores the information so we can just read it back in later, and if we try and run the code again it will not update the code unless the data input has changed. Neat, right?

library(targets)
tar_dir({ # tar_dir() runs code from a temporary directory.
  tar_script({
    library(targets)
    list(
      tar_target(
        terra_rast_example,
        system.file("ex/elev.tif", package = "terra") |> terra::rast()
      )
    )
  })
  tar_make()
  x <- tar_read(terra_rast_example)
  x
})
#> ▶ dispatched target terra_rast_example
#> ● completed target terra_rast_example [1.196 seconds]
#> ▶ ended pipeline [1.825 seconds]
#> 
#> class       : SpatRaster
#> Error: external pointer is not valid

We get an error!

Error: external pointer is not valid

This is a relatively common gotcha moment when using libraries like {terra}. This is due to limitations with its underlying C++ implementation. There are specific ways to write and read these objects. See ?terra for details.

But how do we use {geotargets} to help with this? It helps handle these write and read steps, so you don’t have to worry about them and can use targets as you are used to.

So instead of tar_target(), you use tar_terra_rast() to save a {terra} raster:

library(targets)
tar_dir({ # tar_dir() runs code from a temporary directory.
  tar_script({
    library(targets)
    library(geotargets)
    list(
      tar_terra_rast(
        terra_rast_example,
        system.file("ex/elev.tif", package = "terra") |> terra::rast()
      )
    )
  })
  tar_make()
  x <- tar_read(terra_rast_example)
  x
})
#> ▶ dispatched target terra_rast_example
#> ● completed target terra_rast_example [0.006 seconds]
#> ▶ ended pipeline [0.061 seconds]
#> 
#> class       : SpatRaster 
#> dimensions  : 90, 95, 1  (nrow, ncol, nlyr)
#> resolution  : 0.008333333, 0.008333333  (x, y)
#> extent      : 5.741667, 6.533333, 49.44167, 50.19167  (xmin, xmax, ymin, ymax)
#> coord. ref. : lon/lat WGS 84 (EPSG:4326) 
#> source      : terra_rast_example 
#> name        : elevation 
#> min value   :       141 
#> max value   :       547

Similarly, there are tar_terra_vect() and tar_terra_sprc() for dealing with vector (shapefile) and sprc (collections of rasters). See the README example for more information.

If you’d like to see these functions being used in a more practical context, see the demo-geotargets repository.

What’s next?

We are actively developing {geotargets}, and the next release will focus on adding support for splitting rasters into tiles, preserving SpatRaster metadata, and adding support for {stars}. You can see the full list of issues for more detail on what we are working on.

Thanks

We have recently generously received support from the R Consortium for our project, “{geotargets}: Enabling geospatial workflow management with {targets}”, and so we would like to thank them for their support.

I’d also like to thank Michael Sumner, Anthony North, and Miles McBain for their helpful discussions, as well as Will Landau for writing targets, and being incredibly responsive and helpful to the issues and questions we have asked as we wrote {geotargets}.

An example microbenchmark

Let’s take an example from the naniar package. I’ll give more detail of this story of this optimisation at the end of this section. For the moment, let’s say we want to get the number of missing values in a row of a data frame. We can do something like this:

library(dplyr)
#> 
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:stats':
#> 
#>     filter, lag
#> The following objects are masked from 'package:base':
#> 
#>     intersect, setdiff, setequal, union
my_n_miss <- function(data){
  data |> 
  rowwise() |> 
  mutate(
    n_miss = sum(is.na(c_across(everything())))
  ) |> 
    ungroup()
}

my_n_miss(airquality)
#> # A tibble: 153 × 7
#>    Ozone Solar.R  Wind  Temp Month   Day n_miss
#>    <int>   <int> <dbl> <int> <int> <int>  <int>
#>  1    41     190   7.4    67     5     1      0
#>  2    36     118   8      72     5     2      0
#>  3    12     149  12.6    74     5     3      0
#>  4    18     313  11.5    62     5     4      0
#>  5    NA      NA  14.3    56     5     5      2
#>  6    28      NA  14.9    66     5     6      1
#>  7    23     299   8.6    65     5     7      0
#>  8    19      99  13.8    59     5     8      0
#>  9     8      19  20.1    61     5     9      0
#> 10    NA     194   8.6    69     5    10      1
#> # ℹ 143 more rows

But we can speed this up using rowSums() instead:

new_n_miss <- function(data){
  n_misses <- rowSums(is.na(data))
  data |> 
  mutate(
    n_miss = n_misses
  ) |> 
    as_tibble()
}

new_n_miss(airquality)
#> # A tibble: 153 × 7
#>    Ozone Solar.R  Wind  Temp Month   Day n_miss
#>    <int>   <int> <dbl> <int> <int> <int>  <dbl>
#>  1    41     190   7.4    67     5     1      0
#>  2    36     118   8      72     5     2      0
#>  3    12     149  12.6    74     5     3      0
#>  4    18     313  11.5    62     5     4      0
#>  5    NA      NA  14.3    56     5     5      2
#>  6    28      NA  14.9    66     5     6      1
#>  7    23     299   8.6    65     5     7      0
#>  8    19      99  13.8    59     5     8      0
#>  9     8      19  20.1    61     5     9      0
#> 10    NA     194   8.6    69     5    10      1
#> # ℹ 143 more rows
my_n_miss(airquality)
#> # A tibble: 153 × 7
#>    Ozone Solar.R  Wind  Temp Month   Day n_miss
#>    <int>   <int> <dbl> <int> <int> <int>  <int>
#>  1    41     190   7.4    67     5     1      0
#>  2    36     118   8      72     5     2      0
#>  3    12     149  12.6    74     5     3      0
#>  4    18     313  11.5    62     5     4      0
#>  5    NA      NA  14.3    56     5     5      2
#>  6    28      NA  14.9    66     5     6      1
#>  7    23     299   8.6    65     5     7      0
#>  8    19      99  13.8    59     5     8      0
#>  9     8      19  20.1    61     5     9      0
#> 10    NA     194   8.6    69     5    10      1
#> # ℹ 143 more rows

We can measure the speed using bench::mark():

library(bench)

bm <- mark(
  old = my_n_miss(airquality),
  new = new_n_miss(airquality)
)

bm
#> # A tibble: 2 × 6
#>   expression      min   median `itr/sec` mem_alloc `gc/sec`
#>   <bch:expr> <bch:tm> <bch:tm>     <dbl> <bch:byt>    <dbl>
#> 1 old          23.9ms   23.9ms      41.8   355.5KB    334. 
#> 2 new         347.5µs  384.5µs    2385.     81.8KB     34.8

This runs the code at least twice, and prints out the amount of time it takes to run the code provided on the right hand side of “old” and “new”. But you can name them whatever you want.

Now, it can be kind of hard to see just how much faster this is, if you just look at comparing the times, as the times are given here in…well, actually I’m not sure why our friend from the greek alphabet mu, µ, from the greek alphabet is here, actually? If, like me, you needed to double check the standard measures of order of magnitude wiki page, you might not know that “ms” means milli - or one thousandth, and µ means “micro”, or one millionth. The point is that the new one is many times faster than the old one.

We can do a plot to help see this:

plot(bm)
#> Loading required namespace: tidyr

So we can see that the new one really is a lot faster.

But if I just want to be able to say something like:

It is XX times faster

then we can use the (somewhat unknown?) relative = TRUE option of bench’s S3 method for summary method:

summary(bm, relative = TRUE)
#> # A tibble: 2 × 6
#>   expression   min median `itr/sec` mem_alloc `gc/sec`
#>   <bch:expr> <dbl>  <dbl>     <dbl>     <dbl>    <dbl>
#> 1 old         68.7   62.2       1        4.34     9.62
#> 2 new          1      1        57.0      1        1

And this is great, from this we can see it is about 60 times faster. And that the old approach uses 15 times more memory.

Other packages for microbenchmarking

bench isn’t the only way to measure things! Other ones I’ve enjoyed using in the past are microbenchmark and tictoc. I’ve particularly enjoyed tictoc because you get to do this:

library(tictoc)
tic()
new_n_miss(airquality)
#> # A tibble: 153 × 7
#>    Ozone Solar.R  Wind  Temp Month   Day n_miss
#>    <int>   <int> <dbl> <int> <int> <int>  <dbl>
#>  1    41     190   7.4    67     5     1      0
#>  2    36     118   8      72     5     2      0
#>  3    12     149  12.6    74     5     3      0
#>  4    18     313  11.5    62     5     4      0
#>  5    NA      NA  14.3    56     5     5      2
#>  6    28      NA  14.9    66     5     6      1
#>  7    23     299   8.6    65     5     7      0
#>  8    19      99  13.8    59     5     8      0
#>  9     8      19  20.1    61     5     9      0
#> 10    NA     194   8.6    69     5    10      1
#> # ℹ 143 more rows
toc()
#> 0.01 sec elapsed

Which feels a bit nicer than using system.time():

system.time({
  new_n_miss(airquality)
})
#>    user  system elapsed 
#>   0.001   0.000   0.002

Also, notice that those two times are different? This is why we use benchmarking, to run those checks many times!

End

And that’s it, that’s the blog post. The relative = TRUE option in mark is super neat, and I don’t think many people know about it. Thanks again to Jim Hester for originally creating the bench package.

Imputation functions

I’ve always been a hesitant to include imputation helper functions like impute_mean(), and friends in {naniar}, as they aren’t a great tool to use for imputation. I also really believe the {simputation} R package does a great job at doing much imputation. However, sometimes you do actually want to demonstrate how bad imputing the mean is. Or sometimes you do need to impute the mean. And I’d rather make it easier for people to do this.

This release included some new functions: impute_fixed(), impute_zero(), impute_factor(), and impute_mode(). Notably, these do not implement “scoped variants” which were previously implemented - for example, impute_fixed_if() etc. This is in favour of using the new across() workflow within {dplyr}, which is a bit easier to maintain. This resolves issues #261 and #213.

To demonstrate these imputation functions, let’s create a vector of missing values:

library(naniar)
vec_num <- rnorm(10)
vec_int <- rpois(10, 5)
vec_fct <- factor(LETTERS[1:10])

vec_num <- set_prop_miss(vec_num, 0.4)
vec_int <- set_prop_miss(vec_int, 0.4)
vec_fct <- set_prop_miss(vec_fct, 0.4)

vec_num
#>  [1]         NA  0.9232763         NA  0.5265162         NA         NA
#>  [7]  1.4260531  1.3793303 -0.7855565 -0.5928170
vec_int
#>  [1] NA  5  6  5  5 NA NA  6 NA  6
vec_fct
#>  [1] <NA> B    <NA> D    <NA> F    G    <NA> I    J   
#> Levels: A B C D E F G H I J

We can use impute_fixed() to impute fixed values into the numeric of these:

impute_fixed(vec_num, -999)
#>  [1] -999.0000000    0.9232763 -999.0000000    0.5265162 -999.0000000
#>  [6] -999.0000000    1.4260531    1.3793303   -0.7855565   -0.5928170
impute_fixed(vec_int, -999)
#>  [1] -999    5    6    5    5 -999 -999    6 -999    6

And impute_zero is just a special case of impute_fixed, where the fixed value is 0:

impute_zero(vec_num)
#>  [1]  0.0000000  0.9232763  0.0000000  0.5265162  0.0000000  0.0000000
#>  [7]  1.4260531  1.3793303 -0.7855565 -0.5928170
impute_zero(vec_int)
#>  [1] 0 5 6 5 5 0 0 6 0 6

Similar to impute_mean(), impute_mode() imputes the mode:

impute_mode(vec_num)
#>  [1]  1.0704569  0.9232763  1.0704569  0.5265162  1.0704569  1.0704569
#>  [7]  1.4260531  1.3793303 -0.7855565 -0.5928170
impute_mode(vec_int)
#>  [1] 6 5 6 5 5 6 6 6 6 6

You can’t however use impute_fixed() or impute_zero() on factors, and this doesn’t work for factors, even if it’s a new character.

impute_fixed(vec_fct, -999)
#> Warning in `[<-.factor`(`*tmp*`, is.na(x), value = -999): invalid factor level, NA generated
#>  [1] <NA> B    <NA> D    <NA> F    G    <NA> I    J   
#> Levels: A B C D E F G H I J
impute_zero(vec_fct)
#> Warning in `[<-.factor`(`*tmp*`, is.na(x), value = 0): invalid factor level, NA generated
#>  [1] <NA> B    <NA> D    <NA> F    G    <NA> I    J   
#> Levels: A B C D E F G H I J
impute_fixed(vec_fct, "ZZ")
#> Warning in `[<-.factor`(`*tmp*`, is.na(x), value = "ZZ"): invalid factor level, NA generated
#>  [1] <NA> B    <NA> D    <NA> F    G    <NA> I    J   
#> Levels: A B C D E F G H I J

However, you can use impute_mode():

impute_mode(vec_fct)
#>  [1] I B I D I F G I I J
#> Levels: A B C D E F G H I J

For factors, you can impute a specific value:

impute_factor(vec_fct, value = "ZZ")
#>  [1] ZZ B  ZZ D  ZZ F  G  ZZ I  J 
#> Levels: A B C D E F G H I J ZZ

Think of it like impute_fixed(), but a special case for factors.

Now let’s demonstrate how to do this in a data frame. First we create the data

library(dplyr)

dat <- tibble(
  num = vec_num,
  int = vec_int,
  fct = vec_fct
)

dat
#> # A tibble: 10 × 3
#>       num   int fct  
#>     <dbl> <int> <fct>
#>  1 NA        NA NA   
#>  2  0.923     5 B    
#>  3 NA         6 NA   
#>  4  0.527     5 D    
#>  5 NA         5 NA   
#>  6 NA        NA F    
#>  7  1.43     NA G    
#>  8  1.38      6 NA   
#>  9 -0.786    NA I    
#> 10 -0.593     6 J

You can use it inside mutate like so:

dat %>%
  mutate(
    num = impute_fixed(num, -9999),
    int = impute_zero(int),
    fct = impute_factor(fct, "out")
  )
#> # A tibble: 10 × 3
#>          num   int fct  
#>        <dbl> <dbl> <fct>
#>  1 -9999         0 out  
#>  2     0.923     5 B    
#>  3 -9999         6 out  
#>  4     0.527     5 D    
#>  5 -9999         5 out  
#>  6 -9999         0 F    
#>  7     1.43      0 G    
#>  8     1.38      6 out  
#>  9    -0.786     0 I    
#> 10    -0.593     6 J

Or if you want to impute across all applicable variables with a single function, you could use where like so, to focus on numeric data:

dat %>%
  mutate(
    across(
      .cols = where(is.numeric),
      .fn = impute_zero
    )
  )
#> # A tibble: 10 × 3
#>       num   int fct  
#>     <dbl> <dbl> <fct>
#>  1  0         0 NA   
#>  2  0.923     5 B    
#>  3  0         6 NA   
#>  4  0.527     5 D    
#>  5  0         5 NA   
#>  6  0         0 F    
#>  7  1.43      0 G    
#>  8  1.38      6 NA   
#>  9 -0.786     0 I    
#> 10 -0.593     6 J

dat %>%
  mutate(
    across(
      .cols = where(is.numeric),
      .fn = \(x) impute_fixed(x, -99)
    )
  )
#> # A tibble: 10 × 3
#>        num   int fct  
#>      <dbl> <dbl> <fct>
#>  1 -99       -99 NA   
#>  2   0.923     5 B    
#>  3 -99         6 NA   
#>  4   0.527     5 D    
#>  5 -99         5 NA   
#>  6 -99       -99 F    
#>  7   1.43    -99 G    
#>  8   1.38      6 NA   
#>  9  -0.786   -99 I    
#> 10  -0.593     6 J

Improvements

• Add digit argument to miss_var_summary to help display percentage missing data correctly when there is a very small fraction of missingness - solving #284.

N <- 30000000

df <- tibble(x = rep(NA_real_, N)) %>%
  add_row(x = 0)

df %>% miss_var_summary()
#> # A tibble: 1 × 3
#>   variable   n_miss pct_miss
#>   <chr>       <int>    <num>
#> 1 x        30000000     100.
df %>% miss_var_summary(digits = 6)
#> # A tibble: 1 × 3
#>   variable   n_miss  pct_miss
#>   <chr>       <int> <num:.6!>
#> 1 x        30000000 99.999997

• geom_miss_point() works with shape argument #290

ggplot(
  airquality,
  aes(
    x = Ozone,
    y = Solar.R,
    shape = factor(Month)
  )
) +
  geom_miss_point(size = 4)

• Implemented Date, POSIXct and POSIXlt methods for impute_below() - #158

dat_date <- tibble(
  values = 1:7,
  number = c(111, 112, NA, NA, 108, 150, 160),
  posixct = as.POSIXct(number, origin = "1970-01-01"),
  posixlt = as.POSIXlt(number, origin = "1970-01-01"),
  date = as.Date(number)
  )
dat_date
#> # A tibble: 7 × 5
#>   values number posixct             posixlt             date      
#>    <int>  <dbl> <dttm>              <dttm>              <date>    
#> 1      1    111 1970-01-01 10:01:51 1970-01-01 10:01:51 1970-04-22
#> 2      2    112 1970-01-01 10:01:52 1970-01-01 10:01:52 1970-04-23
#> 3      3     NA NA                  NA                  NA        
#> 4      4     NA NA                  NA                  NA        
#> 5      5    108 1970-01-01 10:01:48 1970-01-01 10:01:48 1970-04-19
#> 6      6    150 1970-01-01 10:02:30 1970-01-01 10:02:30 1970-05-31
#> 7      7    160 1970-01-01 10:02:40 1970-01-01 10:02:40 1970-06-10

date_date_imp <- dat_date %>% 
  mutate(
    number_imp = impute_below(number),
    .after = number
  ) %>% 
  mutate(
    posixct_imp = impute_below(posixct),
    .after = posixct
    ) %>% 
  mutate(
    posixlt_imp = impute_below(posixlt),
    .after = posixlt
  ) %>% 
  mutate(
    date_imp = impute_below(date),
    .after = date
  )

date_date_imp
#> # A tibble: 7 × 9
#>   values number number_imp posixct             posixct_imp        
#>    <int>  <dbl>      <dbl> <dttm>              <dttm>             
#> 1      1    111       111  1970-01-01 10:01:51 1970-01-01 10:01:51
#> 2      2    112       112  1970-01-01 10:01:52 1970-01-01 10:01:52
#> 3      3     NA       103. NA                  1970-01-01 10:01:43
#> 4      4     NA       102. NA                  1970-01-01 10:01:41
#> 5      5    108       108  1970-01-01 10:01:48 1970-01-01 10:01:48
#> 6      6    150       150  1970-01-01 10:02:30 1970-01-01 10:02:30
#> 7      7    160       160  1970-01-01 10:02:40 1970-01-01 10:02:40
#> # ℹ 4 more variables: posixlt <dttm>, posixlt_imp <dttm>, date <date>,
#> #   date_imp <date>

Bug fixes

• Fix bug with all_complete(), which was implemented as !anyNA(x) but should be all(complete.cases(x)).
• Correctly implement any_na() (and any_miss()) and any_complete(). Rework examples to demonstrate workflow for finding complete variables.
• Fix bug with shadow_long() not working when gathering variables of mixed type. Fix involved specifying a value transform, which defaults to character. #314
• Provide replace_na_with(), a complement to replace_with_na() - #129

x <- c(1:5, NA, NA, NA)
x
#> [1]  1  2  3  4  5 NA NA NA
replace_na_with(x, 0)
#> [1] 1 2 3 4 5 0 0 0

dat <- tibble(
  ones = c(1,1,NA),
  twos = c(NA,NA, 2),
  threes = c(NA, NA, NA)
)

dat %>% 
  mutate(
    ones = replace_na_with(ones, 0),
    twos = replace_na_with(twos, -2),
    threes = replace_na_with(threes, -3)
  )
#> # A tibble: 3 × 3
#>    ones  twos threes
#>   <dbl> <dbl>  <dbl>
#> 1     1    -2     -3
#> 2     1    -2     -3
#> 3     0     2     -3

dat %>% 
  mutate(
    across(
      everything(),
      \(x) replace_na_with(x, -99)
    )
  )
#> # A tibble: 3 × 3
#>    ones  twos threes
#>   <dbl> <dbl>  <dbl>
#> 1     1   -99    -99
#> 2     1   -99    -99
#> 3   -99     2    -99

• Fix bug with gg_miss_fct() where it used a deprecated function from forcats - #342

Internal changes

{naniar} now uses cli::cli_abort() and cli::cli_warn() instead of stop() and warning() (#326). Internally in tests we changed to use expect_snapshot() instead of expect_error().

Deprecations

The following functions have been soft deprecated, and will eventually be made defunct in future versions of naniar. shadow_shift() - #193 in favour of impute_below(), and miss_case_cumsum() and miss_var_cumsum() - #257 - in favour of miss_case_summary(data, cumsum = TRUE) and miss_var_summary(data, cumsum = TRUE).

Thanks!

Thanks to everyone for using {naniar}, I’m happy that I’ve got another release out the door and am looking forward to more changes in the future. Especially thanks to everyone who contributed to issues and or pull request for this release, including: @szimmer, @HughParsonage, , @siavash-babaei, @maksymiuks, @jonocarroll, @jzadra, @krlmlr

Learn JS and Rust

I’d like to try and pick up another programming language for 2024. I don’t really have good fluency with other languages outside of R. I can fumble my way through python and HTML/CSS, a wee bit of C++. But I think it’s important and useful to learn a new language to help broaden your own understanding. I’d like to focus on JS (javascript) and Rust.

I think JS because it has such an enormous influence on the web, and the tools we use every day. I’ve taken a nice course by Garrick Aden Buie (I can’t seem to find the course website) in 2020 (one of the last conferences I went to), but I unfortunately didn’t really use JS that year so it’s sort of all fell out of my head.

To learn JS I was thinking I would start with JavaScript30 by Wes Bos, and then move on to You Don’t Know JS by Kyle Simpson. Hopefully I can learn and understand enough to try incorporating JS into R via quarto or htmlwidgets or something.

Rust seems like a better language to use than C++. My experience with the rustlings course was excellent, and the error messages from rust were like, actually understandable? It was bizarre. Plus with the rextender R package you can bundle up your R code just like you do with cpp11 or Rcpp. It’d be cool to build something fast and fun in Rust and then bundle it into an R package.

It’s hard to learn two languages at the same time, so I think I’ll try and do 6 months on JS, and 6 months on Rust.

• January - June: JS
• July - December: Rust

Read some more programming books

I’d really like to read these three books:

• Statistical Rethinking
• The Pragmatic Programmer
• Structure and Interpretation of Computer Programs (potentially even the JS edition)

Hopefully by saying this out aloud I will be able to actually complete this.

Similar to above, I guess I’ll break this up into 3 parts of the year:

• January - April: Statistical Rethinking (~4 chapters a month)
• May - August: The Pragmatic Programmer (2 chapters a month)
• September - December: Structure and Interpretation of Computer Programs (~1 chapter a month. But oh boy is that book massive…)

Make a contribution to R Core

There is now an R Contribution Working Group, which is focussed on encouraging new contributors to R core, with a focus on diversity and inclusion. I’ve joined the slack team, and put the next two meetings in my calendar.

Courses & books

This year I’m going to be helping deliver a few more courses, the rough plan so far is:

• Feb 7: greta course in Canberra
• March 25/26: quarto for scientists (online)
• June: longitudinal data analysis
• Later in the year: another course - functional programming? Missing data?

Accompanying these courses I’d like to publish little living books, like I’ve done with the Rmarkdown for scientists book.

R packages

• Publish and finish conmat
• Regular releases of greta
• Develop and release new pipeline on generalised stacking in insecticide resistance
• Keep my other R packages updated

Work things

The big goals for work this year are related to the above tasks, but the big ticket items for me are:

1. (Finally) release the Tensorflow 2 version of greta
2. Establish regular greta releases, create a greta blog
3. Develop documentation on the internals of greta
4. Develop a culture and community of practice at work with code review
5. Win grants for developing software to solve tricky problems in infectious disease modelling
6. Publish software as JOSS papers

Other misc tech / ideas

• Blog regularly
• Remember how to use switch reliably in R
• Become more proficient with git
• Use lintr in my projects
• Consider doing a livestream of a tidy tuesday data cleaning / EDA
• Condense my internet browsing into an RSS reader
• Focus on getting quality sleep
• Do more running

End

My hope is that I’ll review these goals at the end of 2024. Honestly it’s quite scary to post these plans, and I’m questioning if this is even a good idea to do so publicly. But I think I’ve got the right intention, so let’s leave it at that.

Blog / news posts

Some of these links are things I missed, and other ones are just things I thought were really good. They could be new, they could be a few years old!

Hadley Wickham’s Design Newsletter. I’ve followed Hadley’s work quite closely since 2013, and I was a little alarmed to see I had somehow missed his starting a newsletter about: “designing and implementing quality R code”. I’d seen his book, “tidy design principles” a while back, but it’s super exciting that they are working on it again. I highly recommend the newsletter. It’s also nice to see people engaging with #rstats content in a place that isn’t social media.

The rOpenSci newletter is worth subscribing to, one of my favourite sections is the package development corner, which contains lots of useful links to other cool new R features, blog posts, and more.

Mike Mahoney’s post on classed errors really helped solidify my understanding and the overall benefits to not only the user, but the developer.

King’s Day Speech by Guido van Rossum, the creator of the python language. A nice essay / auto biography. I especially enjoyed this quote:

In reality, programming languages are how programmers express and communicate ideas — and the audience for those ideas is other programmers, not computers. The reason: the computer can take care of itself, but programmers are always working with other programmers, and poorly communicated ideas can cause expensive flops.

They say, “science isn’t science until it’s communicated”, and building on that, I think that good science requires good communication. It doesn’t matter if your idea is world changing, if no one can understand it.

Bob Rudis’s “Daily Driver” post contains an amazing list of things he uses daily from hardware to OS, VPN, RSS readers, browsers, browser extensions, and more.

“Bye, RStudio/Posit!” by Yihui Xie. He’s leaving Posit! It came as a surprise to me. I have a lot of positive feelings about his incredible contributions to R and my career and life, I can’t condense them all here. But I know that his work on knitr, rmarkdown, xaringan, blogdown, and bookdown have all had incredibly positive impacts on how I communicate my work, and I simply don’t think I’d have the career I’ve had without him. Thank you, Yihui.

R Packages

It’s hard to pick R packages to showcase, there are I think more than 20K on CRAN now. Rather than a laundry list of R packages I use, here are some of my favourite R packages that I use.

tidyverse, I’d be lost without it. It is simply an outstanding collection of software for doing data science and statistics in R.

targets, one of my favourite R packages. Will Landau, the developer behind targets is I think perhaps the most responsive and kindest maintainer I’ve ever encountered.

tflow by Miles McBain, an ergonomic package that goes alongside the targets package. Use along with Miles’s other package, fnmate. I think these tips are really useful:

1. Put all your target code in separate functions in R/. Use fnmate to quickly generate function definitions in the right place. Let the plan in _targets.R define the structure of the workflow and use it as a map for your sources. Use ‘jump to function’ to quickly navigate to them.

1. Use a call tar_make() to kick off building your plan in a new R session.

1. Put all your library() calls into packages.R. This way you’ll have them in one place when you go to add sandboxing with renv, packarat, and switchr etc.

1. Take advantage of automation for loading targets targets at the cursor with the ‘load target at cursor’ addin. Or the tflow addin: ‘load editor targets’ to load all targets referred to in the current editor.

autotest by Mark Padgham with rOpenSci. This package does rigorous, automated testing of your R package. Useful to help really kick the tyres of your work.

And not quite an R package, but something from within usethis/rlang as advertised by Garrick Aden Buie, when commenting on my blog post, “Code Smell: Error Handling Eclipse”:

icymi and because it’s not widely publicized, there are type checking functions in {rlang} that you can copy into your package code with usethis::use_standalone() https://github.com/r-lib/rlang/blob/main/R/standalone-types-check.R

This is super useful because it injects selected R functions into your R package. So you can select some standalone functions to import directly into your code with:

use_standalone("https://github.com/r-lib/rlang/blob/main/R/standalone-types-check.R")

Importing code directly is an interesting take on things because normally in R you would import an R package, not copy over code. One benefit of this I think is that it lets you potentially tinker with the imported functions to suit your R package.

Talks / Videos

In part of writing a bit recently, I was reminded by a friend of Jenny Bryan’s UseR! keynote from 2018, Code Smells and Feels - talk materials. It is one of my favourite talks. I was there a few rows from the front when she gave it, but taking time to watch it again has been really great. Jenny is a talented communicator, and I’m going to list a couple of my favourite takeaways from the talk, but I don’t want to start just listing the content out from the talk. These tips will make the most sense alongside Jenny’s talk.

• Write simple condition. Use explaining variables.
• Write functions. A few little functions >> than monster function. Small well-named helper function >> commented code.
• Every if does not need an else, if you exit or return early. There is no else there is only if.
• If your conditions deal with class, it’s time to get object oriented (OO), and use polymorphism.
• switch() is ideal if you need to dispatch different logic, based on a string.
• dplyr::case_when() is ideal if you need to dispatch different data based on data (+logic).

Managing many models with R by Hadley Wickham. I couldn’t find the original keynote that I saw of this, which was at the WOMBAT 2016 conference. The main takeaway from this for me was understanding one of the major benefits of using functions, and understanding how to use map.

The Rise of the Research Software Engineer, by Mike Croucher. A great talk covering a story of becoming a research software engineer, the importance of diversity of people, and technology in a team.

Naming things in code by code aesthetic. I care a lot about naming things, and I really liked these ideas:

• Avoid single letter variables
• Never abbreviate
• Don’t put types in your name (e.g., Icost for Integer Cost), aka “hungarian notation”.
• Put units in variable names. e.g., weight_kg
• Don’t name variables “utils”. Refactor.

Papers

Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author). It’s an absolute classic paper that I found really fundamental to understanding statistical modelling during my PhD. I like it because it introduces the ideas of prediction, and extracting information (inference), which are two really important concepts. It then kind of dunks on the statistics community, saying:

The statistical community has been committed to the almost exclusive use of data models. This commitment has led to irrelevant theory, questionable conclusions, and has kept statisticians from working on a large range of interesting current problems…If our goal as a field is to use data to solve problems, then we need to move away from exclusive dependence on data models and adopt a more diverse set of tools.

Newsletters

I get a lot of these links from trawling through various newsletters and things on twitter and mastodon. Here are some of my current favourites.

Monday Morning Data Science by the Fred Hutch group is great. Once a week, just a couple of links.

Heidi Seibold has a nice newsletter that I enjoy reading: https://heidiseibold.ck.page/

rOpenSci’s newsletter - which you can subscrive to at the bottom of this recent newsletter blog post.

Bob Rudis’s “Daily Drop” newsletter.

Book recommendations

I’ve been meaning to collect together a list of books for programming. Mostly through conversations with Maëlle Salmon, I’ve got a few books I’d like to read. I basically just sifted through Maëlle’s list of book notes and have put a couple here.

• The Pragmatic Programmer - see some reading notes by Maëlle Salmon
• A Philosophy of Software Design - see more reading notes by Maëlle Salmon

End

That’s it. Thanks for reading! What have I missed? Let me know in the comments!

What’s the solution?

This code can be improved by refactoring the errors as check_* functions, something like, check_array_is_square(x).

Show me the difference

To demonstrate this, let’s take a look again at the example code from above, a function that converts a vector into a square matrix:

vector_to_square <- function(data) {
  dims <- sqrt(length(data))
  
  squarable_length <- floor(dims) == dims
  
  if (!squarable_length) {
    cli::cli_abort(
      c(
        "Provided vector is not of a squarable length",
        "{.var x} is of length {.num {length(data)}}",
        "This cannot be represented as a square",
        "Square root of {.var dim(data)} is: {.num {round(dims, 3)}}."
      )
    )
  }
  
  if (!is.numeric(data)) {
    cli::cli_abort(
      c(
        "Provided vector must be {.cls numeric}, not {.cls {class(data)}}",
        "We see that {.run is.numeric(data)} returns {.cls {class(data)}}"
      )
    )
  }
  
  matrix(
    data = data,
    nrow = dims,
    ncol = dims,
    byrow = TRUE
  )
}

Now let’s demonstrate its use, and what the errors look like:

vector_to_square(data = 1:4)
#>      [,1] [,2]
#> [1,]    1    2
#> [2,]    3    4
vector_to_square(data = 1:5)
#> Error in `vector_to_square()`:
#> ! Provided vector is not of a squarable length
#> `x` is of length 5
#> This cannot be represented as a square
#> Square root of `dim(data)` is: 2.236.
vector_to_square(data = LETTERS[1:4])
#> Error in `vector_to_square()`:
#> ! Provided vector must be <numeric>, not <character>
#> We see that `is.numeric(data)` returns <character>

So we have some input checking code, which does two things:

• checks if the inputs can be a square matrix
• checks if inputs are not numeric

While we like these error messages, wrapping up the error messags as functions clarifies the intent of them, and avoids repitition:

check_if_squarable <- function(x) {
  dims <- sqrt(length(x))
  
  squarable_length <- floor(dims) == dims
  
  if (!squarable_length) {
    cli::cli_abort(
      c(
        "Provided vector is not of a squarable length",
        "{.var x} is of length {.num {length(x)}}",
        "This cannot be represented as a square",
        "Square root of {.var dim(x)} is: {.num {round(dims, 3)}}."
      )
    )
  }
}

check_if_not_numeric <- function(x) {
  if (!is.numeric(x)) {
    cli::cli_abort(
      c(
        "Provided vector must be {.cls numeric}, not {.cls {class(x)}}",
        "We see that {.run is.numeric(x)} returns {.cls {class(x)}}"
      )
    )
  }
  
}

These error messages can then be put into the function like so:

vector_to_square <- function(data) {
  check_if_squarable(data)
  check_if_not_numeric(data)
  
  dims <- sqrt(length(data))
  
  matrix(
    data = data,
    nrow = dims,
    ncol = dims,
    byrow = TRUE
  )
}

The error checking code no longer eclipses your intent.

I think the main benefit here is improvement in intent - I don’t have to read some error checking code, I can read the function name check_if_squarable(data) - this tells me it checks if it is squareable, and check_if_not_numeric(data) does what it says on the tin. This means I don’t have to spend time reading error checking code, which can sometimes be quite complex. I can focus on those functions intention. This means I can summarise this function like so:

• Do some checking of the inputs

• Get the dimensions for a square matrix

• Make the matrix, filling in by row

There are a couple of other benefits to this:

• All my checking functions can get re-used in the other work I do
• I can find common cases of checking and improve them.
• I see two checking functions, and it invites me to think about other checks that I might want to perform.

But now the error smells

Ah, but we didn’t check to see what these new errors look like! My friend Adam Gruer pointed out:

might a user get confused when then the error is reported as coming from the check_function rather than the function they called? What are your thoughts?

Indeed, the errors are now different, check it out:

vector_to_square(data = 1:4)
#>      [,1] [,2]
#> [1,]    1    2
#> [2,]    3    4
vector_to_square(data = 1:5)
#> Error in `check_if_squarable()`:
#> ! Provided vector is not of a squarable length
#> `x` is of length 5
#> This cannot be represented as a square
#> Square root of `dim(x)` is: 2.236.
vector_to_square(data = LETTERS[1:4])
#> Error in `check_if_not_numeric()`:
#> ! Provided vector must be <numeric>, not <character>
#> We see that `is.numeric(x)` returns <character>

The error message is about the checking function, check_if_squarable() or check_if_not_numeric(). This could be confusing to the user as the error doesn’t appear to be coming from the vector_to_square() function.

Seeing this also made me realise I’ve seen this problem before and been unsure how to solve it. Thankfully, the rlang team has thought about this, and they have a helpful vignette, “Including function calls in error messages”.

They bring up two really great points. Both of which I hadn’t really thought about solving. I just thought this was the compromise.

The first is how to make sure the input checking function references the function it was called in. This is why we see the error being about check_if_squarable(), and not vector_to_square(), which is what we want.

This is solved by passing the check_ functions a call environment. So we add call = rlang::caller_env() as an argument:

check_if_squarable <- function(x,
                               # Add this
                               call = rlang::caller_env()) {
  dims <- sqrt(length(x))
  
  squarable_length <- floor(dims) == dims
  
  if (!squarable_length) {
    cli::cli_abort(
      message = c(
        "Provided vector is not of a squarable length",
        "{.var x} is of length {.num {length(x)}}",
        "This cannot be represented as a square",
        "Square root of {.var dim(x)} is: {.num {round(dims, 3)}}."
      ),
      # And this
      call = call
    )
  }
}

check_if_not_numeric <- function(x,
                                 # Add this
                                 call = rlang::caller_env()) {
  if (!is.numeric(x)) {
    cli::cli_abort(
      message = c(
        "Provided vector must be {.cls numeric}, not {.cls {class(x)}}",
        "We see that {.run is.numeric(x)} returns {.cls {class(x)}}"
      ),
      # And this
      call = call
    )
  }
  
}

Let’s see what that looks like now:

vector_to_square(data = 1:4)
#>      [,1] [,2]
#> [1,]    1    2
#> [2,]    3    4
vector_to_square(data = 1:5)
#> Error in `vector_to_square()`:
#> ! Provided vector is not of a squarable length
#> `x` is of length 5
#> This cannot be represented as a square
#> Square root of `dim(x)` is: 2.236.
vector_to_square(data = LETTERS[1:4])
#> Error in `vector_to_square()`:
#> ! Provided vector must be <numeric>, not <character>
#> We see that `is.numeric(x)` returns <character>

OK much better, the error now starts with: “Error in vector_to_square()”.

The second problem, which is hauntingly familiar, is to do with supplying argument names. Notice that the error message gives the error in terms of the argument x? Despite our argument to vector_to_square using the argument, data

vector_to_square(data = 1:4)
#>      [,1] [,2]
#> [1,]    1    2
#> [2,]    3    4
vector_to_square(data = 1:5)
#> Error in `vector_to_square()`:
#> ! Provided vector is not of a squarable length
#> `x` is of length 5
#> This cannot be represented as a square
#> Square root of `dim(x)` is: 2.236.
vector_to_square(data = LETTERS[1:4])
#> Error in `vector_to_square()`:
#> ! Provided vector must be <numeric>, not <character>
#> We see that `is.numeric(x)` returns <character>

A small detail, perhaps? But I think it’s actually kind of a big problem! I don’t want my users to have to try and reckon/think about what x is, or have to imagine that I’ve written some fancy code checking functions. I just want them to know where the error came from. In more complex functions this could be even more confusing.

Thankfully, yet again, the rlang team has thought about this, in Input checkers and caller_arg(), they suggest adding arg = rlang::caller_arg(x) to our checking function, which helps capture the name of the argument that is passed to the check functions. Super neat.

Let’s demonstrate this - we change uses of x to using arg in the error message.

check_if_squarable <- function(x,
                               arg = rlang::caller_arg(x),
                               call = rlang::caller_env()) {
  x_len <- length(x)
  dims <- sqrt(x_len)
  
  squarable_length <- floor(dims) == dims
  
  if (!squarable_length) {
    cli::cli_abort(
      message = c(
        "Provided vector is not of a squarable length",
        "{.arg {arg}} is of length {.num {x_len}}",
        "This cannot be represented as a square"
      ),
      call = call
    )
  }
}

check_if_not_numeric <- function(x,
                                 arg = rlang::caller_arg(x),
                                 call = rlang::caller_env()) {
  if (!is.numeric(x)) {
    cli::cli_abort(
      message = c(
        "Provided vector, {.arg {arg}}, must be {.cls numeric}, not {.cls {class(x)}}",
        "We see that {.run is.numeric({.arg {arg}})} returns {.cls {class(x)}}"
      ),
      call = call
    )
  }
  
}

(in rewriting-ing this, I realised I don’t need to include the square root of the length to explain to the user, so I deleted that part of the message)

vector_to_square(data = 1:4)
#>      [,1] [,2]
#> [1,]    1    2
#> [2,]    3    4
vector_to_square(data = 1:5)
#> Error in `vector_to_square()`:
#> ! Provided vector is not of a squarable length
#> `data` is of length 5
#> This cannot be represented as a square
vector_to_square(data = LETTERS[1:4])
#> Error in `vector_to_square()`:
#> ! Provided vector, `data`, must be <numeric>, not <character>
#> We see that `` is.numeric(`data`) `` returns <character>

This now uses data instead of x, which is what we want!

On writing error messages

I do think that writing good error messages is hard. Something that helped me think about this differently was something from the tidyverse style guide on error messages:

An error message should start with a general statement of the problem then give a concise description of what went wrong. Consistent use of punctuation and formatting makes errors easier to parse

They also recommend using cli::cli_abort(), which we used above. I’ve really enjoyed using this over stop, because, well, again, the tidyverse team summarises the reasons well, cli::cli_abort() is good because it:

• Makes it easy to generate bulleted lists.

• Uses glue style interpolation to insert data into the error.

• Supports a wide range of inline markup.

• Provides convenient tools to chain errors together.

• Can control the name of the function shown in the error.

You should read the whole section on error messages, they’ve got great advice on how to write good error messages.

On input checking functions

This type of error message is often called an “input checking function”. There’s a really nice blog post “Checking the inputs of your R functions”, by Hugo Gruson, Sam Abbott, and Carl Pearson, which goes into more detail on the topic. They also recommend a few packages/functions for input checking that are worth checking out, such as checkmate, vec_assert(), vetr, and assertr to list a few.

Some more thoughts

In writing this blog post I’ve now realised that my pattern of creating check_ functions for input checking now also results in a potentially undesirable error for the user where they won’t know where the error has come from. I’ll need to make a note to change this in a lot of my packages.

In the future I’d really like to dive a bit deeper into adding classes or subclasses of errors. Mike Mahoney’s blog post, “Classed conditions from rlang functions” provides a good simple usecase of classed errors. And there’s a tidyverse design page on error constructors that I should read more deeply. I’m pretty sure this would be helpful in things like greta.

Functions are good

I will wrap up by emphasising a point about using functions. Good functions can be individually reasoned with, and used repeatedly across your work. This means you can write them once, use them many times, and only need to make changes to one place, rather than in many. Writing functions helps abstract away details, and helps clarify your code. They are an important building block for writing good code that has content that can be easily reasoned with, and extended.

Thanks

Thank you very much to the maintainers of cli (Gábor Csárdi), and rlang (Lionel Henry), and the team behind these packages. These are hard problems! Thanks to Adam Gruer for asking a great question that made me dive deeper into error checking. Special thanks to Maëlle Salmon for providing very useful comments that improved this post greatly, and thank you in general being a kind source of knowledge.

Is there anything I missed? Other questions / problems to explore in this space? Leave me a comment below if you want to chat about any of this :).

Motivation

Here’s the jam - if you can type faster, then you operate closer to the speed of thought. It’s not just about working faster, as much as it is about being able to clearly express what is in your head. I think when you can clearly express what is in your head, then you put yourself in a good position to enter the wonderful flow state of programming. Flow state, “in the zone”, is where you are completely absorbed, enjoying what you are doing, and things often feel easy and make sense. It’s a good thing. And when you are typing fast, and accurately, you aren’t making too many small stumbling mistakes that would take you out of the flow state. Similarly, when you can use keyboard shortcuts, you are getting more of what is in your head onto the screen fluently, which helps you stay in the flow state.

Typing accurately and fast

Hadley Wickham tweeted about typing speed back in 2019. My friend Miles had written something about this, I think - although I can’t find it on his blog or twitter. But Miles noticed that an improvement in his typing speed helped him stay in the programming flow state. He was already a very talented programmer at this stage, so it’s basically like giving Usain Bolt a pair of roller skates and asking him to race someone downhill.

I don’t think being a fast typist is necessary to be a better programmer, but I think tools that help you work faster and with less resistance are good. I remember changing to a new keyboard, and I felt this friction where I couldn’t put what was in my head onto the screen. I felt like I was slow - like I was walking around with a bunch of ankle weights and a large unwieldy backpack. Like a lot of things in life, taking time to revisit the basics helps you with more advanced skills. I took time every day for a couple of months to work on my typing speed, and it resulted in more accurate, faster typing. Which resulted in faster coding. I ended up ditching the new keyboard, but I kept my typing skills.

There are a lot of options out there for programs to help with typing speed. It’s important to remember to focus on accuracy first. There’s not a lot of benefit to being able to type the wrong and intelligible words really quickly. I liked keybr a lot, and I’ve heard good things about monkeytype.

Also worth noting that I don’t think you need to get a new keyboard to be a faster at typing. I still think that my old mac wireless keyboard (pictured above) is my favourite. I’ve spent a lot of time and money trying to find a good keyboard. I’m not there yet. I think it is a better use of your time to practice typing, than to buy a new keyboard. But hey, you might find that isn’t true for you! Get the keyboard if you want it, I say.

Use keyboard shortcuts

Learn some keyboard shortcuts, it saves not just keystrokes or your time, but it helps you operate a bit closer to the speed of thought. Here are some shorcuts I use that aren’t R related a lot:

• Cmd / Ctrl + C: copy
• Cmd / Ctrl + V: paste
• Cmd / Ctrl + X: cut
• Cmd / Ctrl + Z: undo last action
• Cmd / Ctrl + Shift + Z: reverse the undo
• Cmd / Ctrl + Tab: change application
• Cmd / Ctrl + ~: change window within an application
• Cmd + Space: open up “spotlight” in mac, allowing you to search for files
• Ctrl + Space: open up “Alfred”, which I have configured to look for RStudio project files (see Hadley Wickham’s video on workflow).
• Cmd / Ctrl + Shift + 4: open screenshot tool

I use a screenshot tool, shottr, for Mac that gives you better control of screenshots - opening up a window that lets you crop an image, get colour swatches, save the screenshot to a given directory. It’s really nice, and it’s free.

In a web browser:

• Cmd / Ctrl + Alt + Left / Right: move one browser tab to the left or right
• Cmd / Ctrl + L: move cursor to browser
• Cmd / Ctrl + T: create a new tab
• Cmd / Ctrl + W: close current tab
• Cmd / Ctrl + Shift + T: open up last closed tab
• Cmd / Ctrl + F: Search for a word. I use this across all sorts of apps, PDFs, web browsing, and any code.

(Some of) My favourite shortcuts I use in R:

• Cmd / Ctrl + Enter: run line of code

• Cmd / Ctrl + C: comment/Uncomment current line

• Cmd / Ctrl + Shift + K: knit/render document

• Cmd / Ctrl + Shift + A: format highlighted code (style the code)

• Cmd / Ctrl + D: delete current line

• Alt + Up / Down: yank line up or down

• Cmd / Ctrl + click (on a function): take to function definition

• Cmd / Ctrl + Alt + I: new code chunk

• Cmd / Ctrl + Shift + F: find in files. Search a whole directory, or a specified directory, for files

• Cmd / Ctrl + Alt + Up / Down: copy the above line (or selected lines) down or up. Really useful when you want to copy/paste something. Basically copy/paste, but just that little bit quicker.

• Ctrl + .: go to file/function name. Really useful.

• Cmd / Ctrl + Shift + Enter: run all code in current code chunk or script

• Ctrl + Alt + Up / Down: multiple cursors (!) extend cursor up or down

• Cmd / Ctrl + Left / Right: move the cursor to the end or the beginning of the line

• Alt + Left / Right: move the cursor one word across (combine with Shift to select one/multiple words)

It is a small but very useful benefit with being able to move the cursor across by one word or jumping to the start or end of a line. It means you don’t have to lean on the left or right arrow key to end up where you want - you speed up that process quite a bit!

You can see a quick menu display shortcuts in RStudio with: Alt + Shift + K (press Esc to remove the screen)

And you can see all shortcuts and set your own shortcuts in Rstudio by navigating to “Keyboard Shortcuts Help” (Tools > “Keyboard Shortcuts Help”) (Official notes on customising keyboard shortcuts in the RStudio IDE).

Some packages add keyboard shortcuts as “add-ins” in RStudio, which you can then add as shortcuts using “keyboards shortcuts help”.

Some examples of these add in shortcuts are:

• Render reprex based on selection (I set mine to Cmd / Ctrl + Shift + R) (must have reprex package installed)
• define function at cursor using fnmate (very much worth a look, this has accelerated my functional programming).

Garrick Aden-Buie also has a nice R package that allows you to set other keyboard shortcuts, shrtcts

I really like using my keyboard shorcuts, they help me more fluidly move between tasks, and help me stay in that flow state, that zone.

End

I hope that these are helpful tips! Are there any keyboard shortcuts I missed? Ones that you particularly like? Other points on typing speed or ways to get faster?

Let’s delve a bit deeper

Let’s give examples of all of these things. I’ve realised now as I’ve been writing this that I won’t have time to write the non-coding aspects, so that will be its own blog post. But I thought it might be useful to outline the points above.

From the coding side

weight_mean <- mean(ChickWeight$weight)
weight_sd <- sd(ChickWeight$weight)

x <- mean(ChickWeight$weight)
y <- sd(ChickWeight$weight)

weight_mean <- mean(ChickWeight$weight)
sd_weight <- sd(ChickWeight$weight)

weight_mean <- mean(ChickWeight$weight)
weight_sd <- sd(ChickWeight$weight)

sd_height <- ChickWeight$weight
sd_time <- ChickWeight$Time

# good
sdWeight
sdTime
weightSd
timeSd

# bad
sd_Weight
SD_weight
Long_variable_Name

library(tidyverse)
#> ── Attaching core tidyverse packages ────────────── tidyverse 2.0.0 ──
#> ✔ dplyr     1.1.3     ✔ readr     2.1.4
#> ✔ forcats   1.0.0     ✔ stringr   1.5.0
#> ✔ ggplot2   3.4.4     ✔ tibble    3.2.1
#> ✔ lubridate 1.9.2     ✔ tidyr     1.3.0
#> ✔ purrr     1.0.2     
#> ── Conflicts ──────────────────────────────── tidyverse_conflicts() ──
#> ✖ dplyr::filter() masks stats::filter()
#> ✖ dplyr::lag()    masks stats::lag()
#> ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
diamonds
#> # A tibble: 53,940 × 10
#>    carat cut       color clarity depth table price     x     y     z
#>    <dbl> <ord>     <ord> <ord>   <dbl> <dbl> <int> <dbl> <dbl> <dbl>
#>  1  0.23 Ideal     E     SI2      61.5    55   326  3.95  3.98  2.43
#>  2  0.21 Premium   E     SI1      59.8    61   326  3.89  3.84  2.31
#>  3  0.23 Good      E     VS1      56.9    65   327  4.05  4.07  2.31
#>  4  0.29 Premium   I     VS2      62.4    58   334  4.2   4.23  2.63
#>  5  0.31 Good      J     SI2      63.3    58   335  4.34  4.35  2.75
#>  6  0.24 Very Good J     VVS2     62.8    57   336  3.94  3.96  2.48
#>  7  0.24 Very Good I     VVS1     62.3    57   336  3.95  3.98  2.47
#>  8  0.26 Very Good H     SI1      61.9    55   337  4.07  4.11  2.53
#>  9  0.22 Fair      E     VS2      65.1    61   337  3.87  3.78  2.49
#> 10  0.23 Very Good H     VS1      59.4    61   338  4     4.05  2.39
#> # ℹ 53,930 more rows

diamonds %>%
  mutate(
    price_per_carat = price / carat
  ) %>% 
  group_by(
    cut
    ) %>% 
  summarise(
    price_mean = mean(price_per_carat),
    price_sd = sd(price_per_carat),
    mean_color = mean(color)
  )
#> Warning: There were 5 warnings in `summarise()`.
#> The first warning was:
#> ℹ In argument: `mean_color = mean(color)`.
#> ℹ In group 1: `cut = Fair`.
#> Caused by warning in `mean.default()`:
#> ! argument is not numeric or logical: returning NA
#> ℹ Run `dplyr::last_dplyr_warnings()` to see the 4 remaining warnings.
#> # A tibble: 5 × 4
#>   cut       price_mean price_sd mean_color
#>   <ord>          <dbl>    <dbl>      <dbl>
#> 1 Fair           3767.    1540.         NA
#> 2 Good           3860.    1830.         NA
#> 3 Very Good      4014.    2037.         NA
#> 4 Premium        4223.    2035.         NA
#> 5 Ideal          3920.    2043.         NA

diamonds %>%
  mutate(
    mean_color = mean(color)
  )
#> Warning: There was 1 warning in `mutate()`.
#> ℹ In argument: `mean_color = mean(color)`.
#> Caused by warning in `mean.default()`:
#> ! argument is not numeric or logical: returning NA
#> # A tibble: 53,940 × 11
#>    carat cut       color clarity depth table price     x     y     z mean_color
#>    <dbl> <ord>     <ord> <ord>   <dbl> <dbl> <int> <dbl> <dbl> <dbl>      <dbl>
#>  1  0.23 Ideal     E     SI2      61.5    55   326  3.95  3.98  2.43         NA
#>  2  0.21 Premium   E     SI1      59.8    61   326  3.89  3.84  2.31         NA
#>  3  0.23 Good      E     VS1      56.9    65   327  4.05  4.07  2.31         NA
#>  4  0.29 Premium   I     VS2      62.4    58   334  4.2   4.23  2.63         NA
#>  5  0.31 Good      J     SI2      63.3    58   335  4.34  4.35  2.75         NA
#>  6  0.24 Very Good J     VVS2     62.8    57   336  3.94  3.96  2.48         NA
#>  7  0.24 Very Good I     VVS1     62.3    57   336  3.95  3.98  2.47         NA
#>  8  0.26 Very Good H     SI1      61.9    55   337  4.07  4.11  2.53         NA
#>  9  0.22 Fair      E     VS2      65.1    61   337  3.87  3.78  2.49         NA
#> 10  0.23 Very Good H     VS1      59.4    61   338  4     4.05  2.39         NA
#> # ℹ 53,930 more rows

mean(diamonds$color)
#> Warning in mean.default(diamonds$color): argument is not numeric or logical: returning NA
#> [1] NA

head(diamonds$color)
#> [1] E E E I J J
#> Levels: D < E < F < G < H < I < J

dat <- tibble::tibble(
  weight = rnorm(10, mean = 80, sd = 6),
  height = rnorm(10, mean = 165, sd = 10),
)

dat
#> # A tibble: 10 × 2
#>    weight height
#>     <dbl>  <dbl>
#>  1   78.0   167.
#>  2   76.0   152.
#>  3   81.2   173.
#>  4   71.9   156.
#>  5   88.1   160.
#>  6   80.2   180.
#>  7   67.1   168.
#>  8   82.7   180.
#>  9   81.7   160.
#> 10   81.6   157.

dat$weight_0 <- (dat$weight - mean(dat$weight, na.rm = TRUE)) / sd(dat$weight, na.rm = TRUE)

dat$height_0 <- (dat$height - mean(dat$height, na.rm = TRUE)) / sd(dat$weight, na.rm = TRUE)

mean_center <- function(variable){
  variable - mean(variable, na.rm = TRUE)
}

scale_sd <- function(variable){
  variable / sd(variable, na.rm = TRUE)
}

dat$height %>% 
  mean_center() %>% 
  scale_sd()
#>  [1]  0.1832022 -1.3362374  0.7525296 -0.9848021 -0.4937227  1.4924404
#>  [7]  0.3021602  1.4477151 -0.5125610 -0.8507242

center_scale <- function(variable){
  centered <- mean_center(variable)
  centered_scaled <- scale_sd(centered)
  centered_scaled
}

dat$weight_0 <- (dat$weight - mean(dat$weight, na.rm = TRUE)) / sd(dat$weight, na.rm = TRUE)

dat$height_0 <- (dat$height - mean(dat$height, na.rm = TRUE)) / sd(dat$weight, na.rm = TRUE)

dat
#> # A tibble: 10 × 4
#>    weight height weight_0 height_0
#>     <dbl>  <dbl>    <dbl>    <dbl>
#>  1   78.0   167.   -0.138    0.308
#>  2   76.0   152.   -0.476   -2.25 
#>  3   81.2   173.    0.394    1.27 
#>  4   71.9   156.   -1.18    -1.66 
#>  5   88.1   160.    1.56    -0.831
#>  6   80.2   180.    0.224    2.51 
#>  7   67.1   168.   -1.97     0.508
#>  8   82.7   180.    0.644    2.44 
#>  9   81.7   160.    0.482   -0.862
#> 10   81.6   157.    0.456   -1.43

dat$weight_0 <- center_scale(dat$weight)

dat$height_0 <- center_scale(dat$height)

dat
#> # A tibble: 10 × 4
#>    weight height weight_0 height_0
#>     <dbl>  <dbl>    <dbl>    <dbl>
#>  1   78.0   167.   -0.138    0.183
#>  2   76.0   152.   -0.476   -1.34 
#>  3   81.2   173.    0.394    0.753
#>  4   71.9   156.   -1.18    -0.985
#>  5   88.1   160.    1.56    -0.494
#>  6   80.2   180.    0.224    1.49 
#>  7   67.1   168.   -1.97     0.302
#>  8   82.7   180.    0.644    1.45 
#>  9   81.7   160.    0.482   -0.513
#> 10   81.6   157.    0.456   -0.851

dat$weight_0 <- as.numeric(scale(dat$weight))
dat$height_0 <- as.numeric(scale(dat$height))
dat
#> # A tibble: 10 × 4
#>    weight height weight_0 height_0
#>     <dbl>  <dbl>    <dbl>    <dbl>
#>  1   78.0   167.   -0.138    0.183
#>  2   76.0   152.   -0.476   -1.34 
#>  3   81.2   173.    0.394    0.753
#>  4   71.9   156.   -1.18    -0.985
#>  5   88.1   160.    1.56    -0.494
#>  6   80.2   180.    0.224    1.49 
#>  7   67.1   168.   -1.97     0.302
#>  8   82.7   180.    0.644    1.45 
#>  9   81.7   160.    0.482   -0.513
#> 10   81.6   157.    0.456   -0.851

mean_center <- function(variable){
  browser()
  variable - mean(variable, na.rm = TRUE)
}

debug(mean_center)
mean_center(1:10)
## enter debug mode here, tinker around with the outputs
## then turn off the debugger
undebug(mean_center)

(Not) The End

There’s more to this. I don’t have all the answers. I’m a decent R programmer. But I’m still learning all the time, and I make mistakes a lot. I’m not perfect, far from it! But maybe these things can help you on your journey in improving in R programming.

I didn’t go into as much detail as I would have liked - there are a lot of topics here, what did I miss? What do you think are the ways to get good with R? Write a comment below :)

Milestone one: Outline feature set to implement in the first round (M1)

Summary

Thanks to the R Consortium for providing support for this project, I am very excited to be sharing these updates with the R community, and to be making new improvements to how we think about missing data in R.

Your Turn

Are there things I’ve missed? Other features you’d like to see implemented? Please do drop a comment in this blog post, or write an issue in naniar, or write an issue in visdat.

naniar 1.0.0

I’m very pleased to announce that naniar version 1.0.0 is now on CRAN!

Version 1.0.0 of naniar is to signify that this release is associated with the publication of the associated JSS paper doi:10.18637/jss.v105.i07 (!!!). This paper has been the labour of a lot of effort between myself and Di Cook, and I am very excited to be able to share it.

There is still a lot to do in naniar, and this release does not signify that there are no changes upcoming. It is a 1.0.0 release to establish that this is a stable release, and any changes upcoming will go through a more formal deprecation process.

Here’s a brief description of some of the changes in this release

New things

citation("naniar")
#> 
#> To cite naniar in publications use:
#> 
#>   Tierney N, Cook D (2023). "Expanding Tidy Data Principles to
#>   Facilitate Missing Data Exploration, Visualization and Assessment of
#>   Imputations." _Journal of Statistical Software_, *105*(7), 1-31.
#>   doi:10.18637/jss.v105.i07 <https://doi.org/10.18637/jss.v105.i07>.
#> 
#> A BibTeX entry for LaTeX users is
#> 
#>   @Article{,
#>     title = {Expanding Tidy Data Principles to Facilitate Missing Data Exploration, Visualization and Assessment of Imputations},
#>     author = {Nicholas Tierney and Dianne Cook},
#>     journal = {Journal of Statistical Software},
#>     year = {2023},
#>     volume = {105},
#>     number = {7},
#>     pages = {1--31},
#>     doi = {10.18637/jss.v105.i07},
#>   }

Set missing values with set_n_miss() and set_prop_miss()

These functions allow you to set a random amount of missingness either as a number of values, or as a proportion:

library(naniar)
vec <- 1:10
# different each time
set_n_miss(vec, n = 1)
#>  [1] NA  2  3  4  5  6  7  8  9 10
set_n_miss(vec, n = 1)
#>  [1]  1  2  3  4  5  6  7  8  9 NA

set_prop_miss(vec, prop = 0.2)
#>  [1] NA  2  3 NA  5  6  7  8  9 10
set_prop_miss(vec, prop = 0.6)
#>  [1]  1 NA NA  4 NA NA NA  8  9 NA

I would suggest that these functions are used inside a dataframe. I will provide a few examples below using dplyr. For just one variable, you could set missingness like so:

library(tidyverse)
#> ── Attaching packages ───────────────────────────── tidyverse 1.3.2 ──
#> ✔ ggplot2 3.4.0     ✔ purrr   1.0.1
#> ✔ tibble  3.1.8     ✔ dplyr   1.1.0
#> ✔ tidyr   1.3.0     ✔ stringr 1.5.0
#> ✔ readr   2.1.3     ✔ forcats 1.0.0
#> ── Conflicts ──────────────────────────────── tidyverse_conflicts() ──
#> ✖ dplyr::filter() masks stats::filter()
#> ✖ dplyr::lag()    masks stats::lag()
mtcars_df <- as_tibble(mtcars)

vis_miss(mtcars_df)

mtcars_miss_mpg <- mtcars_df %>% 
  mutate(mpg = set_prop_miss(mpg, 0.5))

vis_miss(mtcars_miss_mpg)

Or add missingness to a few variables:

mtcars_miss_some <- mtcars_df %>% 
  mutate(
    across(
      c(mpg, cyl, disp),
      \(x) set_prop_miss(x, 0.5)
    )
  )

mtcars_miss_some
#> # A tibble: 32 × 11
#>      mpg   cyl  disp    hp  drat    wt  qsec    vs    am  gear  carb
#>    <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
#>  1  NA      NA   NA    110  3.9   2.62  16.5     0     1     4     4
#>  2  21      NA   NA    110  3.9   2.88  17.0     0     1     4     4
#>  3  22.8     4  108     93  3.85  2.32  18.6     1     1     4     1
#>  4  21.4    NA  258    110  3.08  3.22  19.4     1     0     3     1
#>  5  NA      NA   NA    175  3.15  3.44  17.0     0     0     3     2
#>  6  18.1     6   NA    105  2.76  3.46  20.2     1     0     3     1
#>  7  14.3     8   NA    245  3.21  3.57  15.8     0     0     3     4
#>  8  NA      NA  147.    62  3.69  3.19  20       1     0     4     2
#>  9  22.8    NA  141.    95  3.92  3.15  22.9     1     0     4     2
#> 10  19.2     6   NA    123  3.92  3.44  18.3     1     0     4     4
#> # … with 22 more rows

vis_miss(mtcars_miss_some)

Or you can add missingness to all variables like so:

mtcars_miss_all <- mtcars_df %>% 
  mutate(
    across(
      everything(),
      \(x) set_prop_miss(x, 0.5)
    )
  )

mtcars_miss_all
#> # A tibble: 32 × 11
#>      mpg   cyl  disp    hp  drat    wt  qsec    vs    am  gear  carb
#>    <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
#>  1  NA      NA  160    110  3.9   2.62  16.5    NA    NA     4    NA
#>  2  21      NA   NA    110  3.9   2.88  17.0     0     1    NA    NA
#>  3  22.8     4   NA     NA NA    NA     18.6     1    NA     4    NA
#>  4  NA      NA   NA    110 NA    NA     19.4    NA    NA    NA     1
#>  5  NA       8   NA     NA NA     3.44  NA      NA    NA     3     2
#>  6  18.1     6  225     NA NA    NA     20.2     1     0     3     1
#>  7  NA      NA   NA     NA  3.21  3.57  NA       0    NA    NA     4
#>  8  24.4    NA  147.    NA  3.69  3.19  20      NA    NA     4     2
#>  9  NA       4  141.    95  3.92  3.15  22.9    NA     0    NA    NA
#> 10  NA      NA  168.   123  3.92 NA     NA      NA     0     4     4
#> # … with 22 more rows

vis_miss(mtcars_miss_all)

miss_var_summary(mtcars_miss_all)
#> # A tibble: 11 × 3
#>    variable n_miss pct_miss
#>    <chr>     <int>    <dbl>
#>  1 mpg          16       50
#>  2 cyl          16       50
#>  3 disp         16       50
#>  4 hp           16       50
#>  5 drat         16       50
#>  6 wt           16       50
#>  7 qsec         16       50
#>  8 vs           16       50
#>  9 am           16       50
#> 10 gear         16       50
#> 11 carb         16       50

This resolves #298.

Some thank yous

Thank you to everyone who has contributed to this release! Especially the following people: @ddauber, @davidgohel.

I am also excited to announce that I have been supported by the R Consortium to improve how R handles missing values! Through this grant, I will be improving the R packages naniar and visdat. I will be posting more details about this soon, but what this means for you the user is that there will be more updates and improvements to both of these packages in the coming months. Stay tuned.

New Features

options(tidyverse.quiet = TRUE)
library(tidyverse)
library(visdat)

vis_value() - visualise values

The idea of vis_value() is to visualise numeric data, so that you can get a quick idea of the values in your dataset. It does this by scaling all the data between 0 and 1, but it only works with numeric data.

vis_value(mtcars)

It can be fun and interesting to arrange by a variable and then show see how that changes the plot.

vis_cor(mtcars)

mtcars %>% arrange(cyl) %>% vis_value()

Although fair warning that there’s a whole set of statistics/data visualisation that focusses on how to arrange rows and columns - a technique called seriation. For a fun introduction, I’d recommend this lovely blogpost by Nicholas Kruchten. One day I will implement seriation in visdat.

Note that if you use vis_value() on a dataset that isn’t entirely numeric, you will get an error:

vis_value(diamonds)
#> Error in `test_if_all_numeric()` at visdat/R/vis-value.R:33:2:
#> ! Data input can only contain numeric values
#> Please subset the data to the numeric values you would like.
#> `dplyr::select(<data>, where(is.numeric))`
#> Can be helpful here!

vis_binary(dat_bin)

vis_dat(airquality, facet = Month) 

vis_cor(airquality, facet = Month) 

vis_miss(airquality, facet = Month) 

data_vis_dat(airquality)
#> # A tibble: 918 × 4
#>     rows variable valueType value
#>    <int> <chr>    <chr>     <chr>
#>  1     1 Day      integer   41   
#>  2     1 Month    integer   190  
#>  3     1 Ozone    integer   7.4  
#>  4     1 Solar.R  integer   67   
#>  5     1 Temp     integer   5    
#>  6     1 Wind     numeric   1    
#>  7     2 Day      integer   36   
#>  8     2 Month    integer   118  
#>  9     2 Ozone    integer   8    
#> 10     2 Solar.R  integer   72   
#> # … with 908 more rows

data_vis_miss(airquality)
#> # A tibble: 918 × 4
#>     rows variable valueType value
#>    <int> <chr>    <chr>     <chr>
#>  1     1 Day      FALSE     FALSE
#>  2     1 Month    FALSE     FALSE
#>  3     1 Ozone    FALSE     FALSE
#>  4     1 Solar.R  FALSE     FALSE
#>  5     1 Temp     FALSE     FALSE
#>  6     1 Wind     FALSE     FALSE
#>  7     2 Day      FALSE     FALSE
#>  8     2 Month    FALSE     FALSE
#>  9     2 Ozone    FALSE     FALSE
#> 10     2 Solar.R  FALSE     FALSE
#> # … with 908 more rows

data_vis_cor(airquality)
#> # A tibble: 36 × 3
#>    row_1   row_2     value
#>    <chr>   <chr>     <dbl>
#>  1 Ozone   Ozone    1     
#>  2 Ozone   Solar.R  0.348 
#>  3 Ozone   Wind    -0.602 
#>  4 Ozone   Temp     0.698 
#>  5 Ozone   Month    0.165 
#>  6 Ozone   Day     -0.0132
#>  7 Solar.R Ozone    0.348 
#>  8 Solar.R Solar.R  1     
#>  9 Solar.R Wind    -0.0568
#> 10 Solar.R Temp     0.276 
#> # … with 26 more rows

airquality %>% group_by(Month) %>% data_vis_dat()
#> # A tibble: 765 × 5
#> # Groups:   Month [5]
#>    Month  rows variable valueType value
#>    <int> <int> <chr>    <chr>     <chr>
#>  1     5     1 Day      integer   41   
#>  2     5     1 Ozone    integer   190  
#>  3     5     1 Solar.R  integer   7.4  
#>  4     5     1 Temp     integer   67   
#>  5     5     1 Wind     numeric   1    
#>  6     5     2 Day      integer   36   
#>  7     5     2 Ozone    integer   118  
#>  8     5     2 Solar.R  integer   8    
#>  9     5     2 Temp     integer   72   
#> 10     5     2 Wind     numeric   2    
#> # … with 755 more rows

Missing values show up in list columns

vis_dat() vis_miss() and vis_guess() now render missing values in list-columns. Let’s demonstrate this with the star_wars dataset from dplyr, which has a few list columns.

starwars
#> # A tibble: 87 × 14
#>    name        height  mass hair_…¹ skin_…² eye_c…³ birth…⁴ sex   gender homew…⁵
#>    <chr>        <int> <dbl> <chr>   <chr>   <chr>     <dbl> <chr> <chr>  <chr>  
#>  1 Luke Skywa…    172    77 blond   fair    blue       19   male  mascu… Tatooi…
#>  2 C-3PO          167    75 NA      gold    yellow    112   none  mascu… Tatooi…
#>  3 R2-D2           96    32 NA      white,… red        33   none  mascu… Naboo  
#>  4 Darth Vader    202   136 none    white   yellow     41.9 male  mascu… Tatooi…
#>  5 Leia Organa    150    49 brown   light   brown      19   fema… femin… Aldera…
#>  6 Owen Lars      178   120 brown,… light   blue       52   male  mascu… Tatooi…
#>  7 Beru White…    165    75 brown   light   blue       47   fema… femin… Tatooi…
#>  8 R5-D4           97    32 NA      white,… red        NA   none  mascu… Tatooi…
#>  9 Biggs Dark…    183    84 black   light   brown      24   male  mascu… Tatooi…
#> 10 Obi-Wan Ke…    182    77 auburn… fair    blue-g…    57   male  mascu… Stewjon
#> # … with 77 more rows, 4 more variables: species <chr>, films <list>,
#> #   vehicles <list>, starships <list>, and abbreviated variable names
#> #   ¹​hair_color, ²​skin_color, ³​eye_color, ⁴​birth_year, ⁵​homeworld

glimpse(starwars)
#> Rows: 87
#> Columns: 14
#> $ name       <chr> "Luke Skywalker", "C-3PO", "R2-D2", "Darth Vader", "Leia Or…
#> $ height     <int> 172, 167, 96, 202, 150, 178, 165, 97, 183, 182, 188, 180, 2…
#> $ mass       <dbl> 77.0, 75.0, 32.0, 136.0, 49.0, 120.0, 75.0, 32.0, 84.0, 77.…
#> $ hair_color <chr> "blond", NA, NA, "none", "brown", "brown, grey", "brown", N…
#> $ skin_color <chr> "fair", "gold", "white, blue", "white", "light", "light", "…
#> $ eye_color  <chr> "blue", "yellow", "red", "yellow", "brown", "blue", "blue",…
#> $ birth_year <dbl> 19.0, 112.0, 33.0, 41.9, 19.0, 52.0, 47.0, NA, 24.0, 57.0, …
#> $ sex        <chr> "male", "none", "none", "male", "female", "male", "female",…
#> $ gender     <chr> "masculine", "masculine", "masculine", "masculine", "femini…
#> $ homeworld  <chr> "Tatooine", "Tatooine", "Naboo", "Tatooine", "Alderaan", "T…
#> $ species    <chr> "Human", "Droid", "Droid", "Human", "Human", "Human", "Huma…
#> $ films      <list> <"The Empire Strikes Back", "Revenge of the Sith", "Return…
#> $ vehicles   <list> <"Snowspeeder", "Imperial Speeder Bike">, <>, <>, <>, "Imp…
#> $ starships  <list> <"X-wing", "Imperial shuttle">, <>, <>, "TIE Advanced x1",…

vis_dat(starwars)

vis_miss(starwars)

vis_guess(starwars)

As you can see, lists are now displayed in the visualisation. Unfortunately vis_guess has trouble guessing lists, but that is a limitation due to how it guesses variable types.

Thank you to github user cregouby for adding this in #138.

long_data <- tibble(
  really_really_long_name = c(NA, NA, 1:8),
  very_quite_long_name = c(-1:-8, NA, NA),
  this_long_name_is_something_else = c(NA, NA,
                                       seq(from = 0, to = 1, length.out = 8))
)

vis_miss(long_data)