diff --git a/01-big-shiny.Rmd b/01-big-shiny.Rmd index 7bfb2d0..a67f690 100644 --- a/01-big-shiny.Rmd +++ b/01-big-shiny.Rmd @@ -10,7 +10,7 @@ If you are reading this page, chances are you already know what a `{shiny}` appl The beauty of `{shiny}` [@R-shiny] is that it makes it easy for someone already familiar with R to create a small app in a matter of hours. With small and minimal `{shiny}` apps, no knowledge of HTML (HyperText Markup Language), CSS (Cascading Style Sheets) or JavaScript is required, and you do not have to think about technical elements that usually come with web applications—for example, you do not have to think about the port the application is served on: `{shiny}` picks one for you.[^big-shiny-1] Same goes for serving external dependencies: the application comes with its set of CSS and JavaScript dependencies that a common `{shiny}` developer does not need to worry about. -And that is probably one of the main reasons why this package has become so successful over the years—**with very little training, you can rapidly create a proof-of-concept (PoC) for a data product, showcase an algorithm, or present your results in an elegant and accessible user interfaces**. +And that is probably one of the main reasons why this package has become so successful over the years—**with very little training, you can rapidly create a proof-of-concept (PoC) for a data product, showcase an algorithm, or present your results in an elegant and accessible user interface**. [^big-shiny-1]: Of course you can specify one if you need to, but by default the package picks one. @@ -36,7 +36,7 @@ Ready to start engineering production-grade `{shiny}` apps? Building a `{shiny}` application seems quite straightforward when it comes to small prototypes or proof of concepts: after a few hours of practice and documentation reading, most R developers can have a small working application.\ But things change when your application reaches "the cliff of complexity",[^big-shiny-2] i.e. that moment when the application reaches a state when it can be qualified as "complex". -[^big-shiny-2]: We borrow this term from Charity Major, as heard in *Test in Production with Charity Majors* CoRecursive #019, _Aug 31, 2018_. +[^big-shiny-2]: We borrow this term from Charity Majors, as heard in *Test in Production with Charity Majors* CoRecursive #019, _Aug 31, 2018_. But what do we mean by complexity? Getting a clear definition is not an easy task [^big-shiny-3] as it very much depends on who is concerned and who you are talking to. @@ -73,7 +73,7 @@ One of the goals of this book is to present a methodology and toolkit that will Customers and end users see complexity as *interface complexity*. Interface complexity can be driven by a lot of elements, for example, the probability of making an error while using the app, the difficulty in understanding the logical progression in the app, the presence of unfamiliar behavior or terms, visual distractions, etc. -This book will also bring you strategy to help you cope with the need for simplification when it comes to designing an interface. +This book will also bring you strategies to help you cope with the need for simplification when it comes to designing an interface. ### Balancing complexities @@ -119,7 +119,7 @@ But before that, let's dive into code complexity. #### A. Codebase size {.unnumbered} -The total number of lines of code, and the number of files, can be good clue of potential complexity, but only if used as an order of magnitude (for example, a 10,000-line codebase is potentially more complex than a 100-line codebase), but should not be relied on if used strictly, even more if you try to reduce the number of lines by sacrificing code readability. +The total number of lines of code, and the number of files, can be a good clue of potential complexity, but only if used as an order of magnitude (for example, a 10,000-line codebase is potentially more complex than a 100-line codebase), but should not be relied on if used strictly, even more if you try to reduce the number of lines by sacrificing code readability. R is very permissive when it comes to indentation and line breaks, and, unlike JavaScript or CSS, it is generally not minified.[^big-shiny-6] In R, the number of lines of code depends on your coding style and the packages you are using. For example, the `{tidyverse}` [@tidyverse2019] style guide encourages the use of `%>%` (called "pipe"), with one function by line, producing more lines in the end code: "`%>%` should always have a space before it, and should usually be followed by a new line" ([tidyverse style guide](https://style.tidyverse.org/pipes.html){target="_blank"}). @@ -161,13 +161,13 @@ Six lines of code for something that could also be written in one line. iris[1:5, "Species"] ``` -In other words, using this kind of writing style can make the codebase larger in term of lines, without really adding complexity to the general program. +In other words, using this kind of writing style can make the codebase larger in terms of lines, without really adding complexity to the general program. Another drawback of this metric is that it focuses on numbers instead of readability, and in the long run, yes, readability matters. As noted in *The Art of Unix Programming*, "Pressure to keep the codebase size down by using extremely dense and complicated implementation techniques can cause a cascade of implementation complexity in the system, leading to an un-debuggable mess" [@ericraymond2003]. Still, this metric can be useful to reinforce what you have learned from other metrics. -It is rather unlikely that you will find this "extreme" coding style we showed above, and even if it might not make sense to compare two codebases that just differ by 1% or 2 % of lines of code, it is very likely that a codebase which is ten, one hundred, one thousand times larger is a more complex software. +It is rather unlikely that you will find this "extreme" coding style we showed above, and even if it might not make sense to compare two codebases that just differ by 1% or 2% of lines of code, it is very likely that a codebase which is ten, one hundred, one thousand times larger is more complex software. Another good metric related to code complexity is the number of files in the project: R developers tend to split their functions into several files, so the more files you will find in a project, the larger the codebase is. And numerous files can also be a sign of maintenance complexity, as it may be harder to reason about an app logic that is split into several files than about something that fits into one linear code inside one file.[^big-shiny-8] @@ -248,7 +248,7 @@ structure(list(pkg = c("attempt", "shiny"), files = c(64L, 736L Here, with these two metrics, we can safely assume that `{shiny}` is a more complex package than `{attempt}`. -If you want to compute the same prefix for a local package/repository, the `cloc_pkg()` function can be used. +If you want to compute the same metric for a local package/repository, the `cloc_pkg()` function can be used. For example, here is how to compute the cloc metric for the `{hexmake}` application: ```{r 01-big-shiny-8, echo = FALSE} @@ -303,8 +303,8 @@ hexmake_cloc ``` -One thing that this package also allows is counting the number of lines of commented code: it's usually a good sign to see that a package has comments in its codebase, as it will allow to work more safely in the future, provided that this metric doesn't reveal that large portions of the application are "commented code" (as opposed to "code comments"). -For example, here we can see that `{hemake}` has `r dplyr::filter(hexmake_cloc, language == "R") %>% dplyr::pull(loc)` lines of R code, which come with `r dplyr::filter(hexmake_cloc, language == "R") %>% dplyr::pull(comment_lines)` lines of code comments. +One thing that this package also allows is counting the number of lines of commented code: it's usually a good sign to see that a package has comments in its codebase, as it will allow you to work more safely in the future, provided that this metric doesn't reveal that large portions of the application are "commented code" (as opposed to "code comments"). +For example, here we can see that `{hexmake}` has `r dplyr::filter(hexmake_cloc, language == "R") %>% dplyr::pull(loc)` lines of R code, which come with `r dplyr::filter(hexmake_cloc, language == "R") %>% dplyr::pull(comment_lines)` lines of code comments. #### B. Cyclomatic complexity {.unnumbered} @@ -324,7 +324,7 @@ knitr::include_graphics("img/controlflow.png") The complexity number is then computed by taking the number of nodes, subtracting the number of edges, and adding twice the number of connected components of this graph. The algorithm is then $M = E − N + 2P$, where $M$ is the measure, $E$ the number of edges, $N$ the number of nodes and $2P$ is twice the number of connected components. -We will not go deep into this topic, as there are a lot things going on in this computation and you can find much documentation about this online. +We will not go deep into this topic, as there are a lot of things going on in this computation and you can find much documentation about this online. Please refer to the bibliography for further readings about the theory behind this measurement. In R, the cyclomatic complexity can be computed using the `{cyclocomp}` [@R-cyclocomp] package. @@ -341,7 +341,7 @@ install.packages("cyclocomp") ``` The `{cyclocomp}` package comes with three main functions: `cyclocomp()`, `cyclocomp_package()`, and `cyclocomp_package_dir()`. -While developing your application, the one you will be interested in is `cyclocomp_package_dir()`: building successful shiny apps with the `{golem}` framework means you will be building your app as a package (we will get back on that later). +While developing your application, the one you will be interested in is `cyclocomp_package_dir()`: building successful shiny apps with the `{golem}` framework means you will be building your app as a package (we will get back to that later). Here is, for example, the cyclomatic complexity of the default golem template (assuming it is located in a `golex/` subdirectory): @@ -417,7 +417,7 @@ You might have heard this saying: "if you copy and paste a piece of code twice, Indeed, splitting code into smaller pieces lowers the local cyclomatic complexity, as smaller functions have lower cyclomatic complexity. But that is just at a local level, and it can be a suboptimal option: having a very large number of functions calling each other can make it harder to navigate through the codebase. -In the end of the day, splitting into smaller functions is not a magic solution because: +At the end of the day, splitting into smaller functions is not a magic solution because: - the global complexity of the app is not lowered by splitting things into pieces (just local complexity) and - The deeper the call stack, the harder it can be to debug. @@ -470,7 +470,7 @@ And run the metric for `{golem}`, ``` ```{r 01-big-shiny-27, cache=TRUE, warning=FALSE, eval = FALSE} -# Using this function with{golem} +# Using this function with {golem} frame_metric("golem") ``` @@ -512,11 +512,11 @@ attachment::att_from_description("golex/DESCRIPTION") ``` -Some important metrics to watch there are as follow: +Some important metrics to watch there are as follows: - Test coverage: the more the better, as a large code coverage should imply that bugs are more easily caught. -- The number of downloads: a largely downloaded package will likely be less prone to bug, as it will be used by a large user base. -- Number of dependencies: the more a package has dependencies, the more likely it is that at some point it time, something in the dependency graph will break. +- The number of downloads: a largely downloaded package will likely be less prone to bugs, as it will be used by a large user base. +- Number of dependencies: the more a package has dependencies, the more likely it is that at some point in time, something in the dependency graph will break. - Dates of first publish on CRAN, last publish, and updates: a package actively maintained is a good sign.[^big-shiny-10] [^big-shiny-10]: Even if this is not an absolute rule, some packages haven't been updated for a long time but are still completely reliable. @@ -544,7 +544,7 @@ And this is not necessarily a bad thing! `{shiny}` apps can definitely be used to implement production-grade [^big-shiny-11] software, but production-grade software implies production-grade software engineering. To make your project a success, you need to use tools that reduce the complexity of your app and ensure that your app is resilient to aging. -[^big-shiny-11]: By production-grade, we mean a software that can be used in a context where people use it for doing their job, and where failures or bugs have real-life consequences. +[^big-shiny-11]: By production-grade, we mean software that can be used in a context where people use it for doing their job, and where failures or bugs have real-life consequences. In other words, production-grade `{shiny}` apps require working with a software engineering mindset, which is not always an easy task in the R world: many R developers have learned this language as a tool for doing data analysis, building models, and making statistics; not really as a tool for building software. @@ -600,4 +600,4 @@ Once the app is out, it is successful if it can **exist in the long run, with al And this, again, is hard to do without effective planning and efficient engineering. [^big-shiny-12]: In fact, this new person might simply be you, a month from now. - And *"You'll be there in the future too, maintaining code you may have half forgotten under the press of more recent projects. When you design for the future, the sanity you save may be your own.* [@ericraymond2003]. + And *"You'll be there in the future too, maintaining code you may have half forgotten under the press of more recent projects. When you design for the future, the sanity you save may be your own."* [@ericraymond2003]. diff --git a/02-planning-ahead.Rmd b/02-planning-ahead.Rmd index bf0fa96..10eeccb 100644 --- a/02-planning-ahead.Rmd +++ b/02-planning-ahead.Rmd @@ -31,7 +31,7 @@ The larger the codebase, the harder it is to untangle everything and make it wor In this book, we will present a framework called `{golem}`, which is a toolbox for building production-grade `{shiny}` applications. Even if `{golem}` is focused on production, there is no reason not to use it for your proof of concepts: starting a new `{golem}` project is relatively straightforward, and even if you do not use the advanced features, you can use it for very small apps. The benefit of starting straight inside a `{golem}` application really outweighs the cost. -We hear a lot the question "When should I switch to `{golem}`?" The answer is simple: do not switch to `{golem}`, start with it. +We often hear the question "When should I switch to `{golem}`?" The answer is simple: do not switch to `{golem}`, start with it. That way, you are getting ready for complexity, and if, one day, you need to turn this small app into a production app, the foundations are there. ### Develop with the KISS principle @@ -40,7 +40,7 @@ That way, you are getting ready for complexity, and if, one day, you need to tur > > *KISS principle, Wikipedia article* () -The KISS principle, as "Keep It Simple, Stupid", should drive the implementation of features in the application to allow anyone in the future, including original developers, to take over on the development. +The KISS principle, as "Keep It Simple, Stupid", should drive the implementation of features in the application to allow anyone in the future, including original developers, to take over the development. The story behind this principle is supposed to be that Kelly Johnson, lead engineer at the Lockheed Skunk Works, gave his workers a set of very common tools and said that every airplane should be repairable with these tools, and these tools only, so that repairing an aircraft should be possible for any average engineer. @@ -48,7 +48,7 @@ This should be a principle to keep in mind when building applications. Indeed, large-scale `{shiny}` projects can lead to many people working on the codebase, for a long period of time. **A large team means a variety of skills**, with some common ground in `{shiny}` development, but potentially various levels when it comes to R, web development, or production engineering. When choosing how and what to implement, **try to make a rule to go for the simplest solution**,[^planning-ahead-1] *i.e.* the one that any common `{shiny}` developer would be able to understand and maintain. -If you go for an exotic solution or a complex technology, be sure that you are doing it for a good reason: unknown or hard-to-grasp technology reduces the chance of finding someone that will be able to maintain that piece of code in the future, and reduce the smoothness of collaboration, as "*Code you can easily comprehend elevates absolutely everyone on your team, no matter their tenure or experience level*" [@lemaire2020]. +If you go for an exotic solution or a complex technology, be sure that you are doing it for a good reason: unknown or hard-to-grasp technology reduces the chance of finding someone that will be able to maintain that piece of code in the future, and reduces the smoothness of collaboration, as "*Code you can easily comprehend elevates absolutely everyone on your team, no matter their tenure or experience level*" [@lemaire2020]. [^planning-ahead-1]: Which might not be the most "elegant" solution, but production code requires pragmatism. @@ -120,7 +120,7 @@ If you follow this book's workflow, this person will first create a `{golem}` pr Once the skeleton of the app is created, this person in charge will list all the things that have to be done. We strongly suggest that you use `Git` with a graphical interface (GitLab, GitHub, Bitbucket, etc.) as the graphical interface to help you manage the project. These tasks are defined as issues, and will be closed during development. -These interfaces can also be used to set continuous integration. +These interfaces can also be used to set up continuous integration. If the team follows a `git flow` (described in Chapter \@ref(version-control)), the manager will also be in charge of reviewing and accepting the pull/merge requests to the main `dev` branch if they solve the associated issues. @@ -130,7 +130,7 @@ Do not worry if this sounds like a foreign language to you, we will get back to **Developers will focus on small features**. If the person in charge has correctly separated the work between developers of the team, they will be focusing on one or more parts of the application, but do not need to know every single bit of what the application is doing. -In a perfect world, the application is split in various `{shiny}` modules, one module equals one file, and each member of the team will be assigned to the development of one or more modules. +In a perfect world, the application is split into various `{shiny}` modules, one module equals one file, and each member of the team will be assigned to the development of one or more modules. It is simpler to work in this context where one developer is assigned to one module, although we know that in reality it may be a little more complex, and several members of the team might go back and forth working on a common module. But the person in charge will be there to help make all the pieces fit together. diff --git a/03-structure.Rmd b/03-structure.Rmd index 32ce1f3..fe2309a 100644 --- a/03-structure.Rmd +++ b/03-structure.Rmd @@ -78,7 +78,7 @@ The good news is that using the R package structure helps you leverage the commo - A `README` file that you will put at the root of your package, which will document how to install the package, and some information about how to use the package. Note that in many cases developers go for a `.md` file (short for markdown) because this format is automatically rendered on services like GitHub, GitLab, or any other main version control system. -- `Vignettes` are longer-form documentation that explains in more depth how to use your app. They are also useful if you need to detail the core functions of the application using a static document, notably for prototyping and/or for exchanging with the client. We will get back to `Vignettes` in Chapter \@ref(building-ispum-app) when we will talk about prototyping. +- `Vignettes` are longer-form documentation that explains in more depth how to use your app. They are also useful if you need to detail the core functions of the application using a static document, notably for prototyping and/or for exchanging with the client. We will get back to `Vignettes` in Chapter \@ref(building-ispum-app) when we talk about prototyping. - Function documentation. Every function in your package should come with its own documentation, even if it is just for your future self. "Exported" functions, the ones which are available once you run `library(myapp)`, should be fully documented and will be listed in the package help page. Internal functions need less documentation, but documenting them is the best way to be sure you can come back to the app in a few months and still know why things are the way they are, what the pieces of the apps are used for, and how to use these functions.[^structure-1] @@ -96,7 +96,7 @@ The other thing we need for our application to be successful in the long run is Testing production apps is a broad question that we will come back to in another chapter, but let's talk briefly about why using a package structure helps with testing. -Frameworks for package testing are robust and widely documented in the R world, and if you choose to embrace the "`{shiny}` app as a package" structure, you do not have to put in any extra-effort for testing your application back-end: use a canonical testing framework like `{testthat}` [@R-testthat]. +Frameworks for package testing are robust and widely documented in the R world, and if you choose to embrace the "`{shiny}` app as a package" structure, you do not have to put in any extra effort for testing your application back-end: use a canonical testing framework like `{testthat}` [@R-testthat]. Learning how to use it is not the subject of this chapter, so feel free to refer to the documentation, and see also Chapter 5 of the [workshop: "Building a Package that Lasts"](https://speakerdeck.com/colinfay/building-a-package-that-lasts-erum-2018-workshop?slide=107). We will come back to testing in Chapter 11: "Build Yourself a Safety Net". @@ -252,7 +252,7 @@ We assume that you know the saying that "if you copy and paste something more th In a `{shiny}` application, how can we refactor a partially repetitive piece of code so that it is reusable? Yes, you guessed right: using shiny modules. -**`{shiny}` modules aim at three things: simplify "id" namespacing, split the codebase into a series of functions, and allow UI/Server parts of your app to be reused. Most of the time, modules are used to do the two first. In our case, we could say that 90% of the modules we write are never reused;[^structure-2] they are here to allow us to split the codebase into smaller, more manageable pieces**. +**`{shiny}` modules aim at three things: simplify "id" namespacing, split the codebase into a series of functions, and allow UI/Server parts of your app to be reused. Most of the time, modules are used to do the first two. In our case, we could say that 90% of the modules we write are never reused;[^structure-2] they are here to allow us to split the codebase into smaller, more manageable pieces**. [^structure-2]: Most of the time, pieces / panels of the app are too unique to be reused elsewhere. @@ -371,17 +371,17 @@ ns("choice") And here it is, our namespaced `id`! -Each call to a module with `choice_server()` requires a different `id` argument that will allow creating various internal namespaces to prevent from id conflicts.[^structure-3] +Each call to a module with `choice_server()` requires a different `id` argument that will allow creating various internal namespaces to prevent id conflicts.[^structure-3] Then you can have as many `validate` ids as you want in your whole app, as long as this input has a unique id inside your module. [^structure-3]: Well, of course you can still have inner module id conflicts, but they are easier to avoid, detect, and fix. ##### Note for `{shiny}` \< 1.5.0 {.unnumbered} -Released on 2020-06-23, the version 1.5.0 of `{shiny}` introduced a new way to write shiny modules, using a new function called `moduleServer()`. +Released on 2020-06-23, version 1.5.0 of `{shiny}` introduced a new way to write shiny modules, using a new function called `moduleServer()`. This new function was introduced to make the couple `ui_function` / `server_function` more obvious, where the old version used to require a `callModule(server_function, id)` call. This `callModule` notation is still valid (at least at the time of writing these lines), but we chose to go for the `moduleServer()` notation in this book. -Most of the applications that are used as examples in this book have been built before this new function though, so when you will read their code, you will find the `callModule` implementation. +Most of the applications that are used as examples in this book have been built before this new function though, so when you read their code, you will find the `callModule` implementation. #### B. Passing arguments to your modules {.unnumbered} @@ -421,7 +421,7 @@ Figure \@ref(fig:03-structure-6) is a screenshot of this application. This application contains 6 tabs, 4 of them being pretty much alike: a side bar with inputs, a main panel with a button, and the plot. This plot can be, depending on the tab, a scatterplot, a histogram, a boxplot, or a barplot. -This is a typical case where you should reuse modules: if several parts are relatively similar, it is easier to bundle it inside a reusable module, and condition the UI/server with function arguments. +This is a typical case where you should reuse modules: if several parts are relatively similar, it is easier to bundle them inside a reusable module, and condition the UI/server with function arguments. (ref:tidytuesdayappcap) Snapshot of the `{tidytuesday201942}` `{shiny}` application. @@ -583,7 +583,7 @@ app_server <- function(input, output, session) { ### Communication between modules -One of the hardest part of using modules is sharing data across them. +One of the hardest parts of using modules is sharing data across them. There are at least three approaches: - Returning a `reactive` function @@ -597,7 +597,7 @@ One common approach is to return a `reactive` function from one module**, and pa Here is an example that illustrates this pattern. ```{r 03-structure-11, eval = FALSE} -# Module 1, which will allow to select a number +# Module 1, which will allow selecting a number choice_ui <- function(id) { ns <- NS(id) tagList( @@ -663,7 +663,7 @@ shinyApp(app_ui, app_server) ``` This strategy works well, but for large `{shiny}` apps it might be hard to handle large lists of reactive outputs / inputs and to keep track of how things are organized. -It might also create some reactivity issues, as a lot of `reactive` function calls is harder to control, or lead to too much computation from the server. +It might also create some reactivity issues, as a lot of `reactive` function calls are harder to control, or lead to too much computation from the server. #### B. The "stratégie du petit r" {.unnumbered} @@ -675,7 +675,7 @@ Below, we create a "global" (in the sense that it is initiated at the top of the It will then go through all modules, passed as a function argument. ```{r 03-structure-12, eval = FALSE} -# Module 1, which will allow to select a number +# Module 1, which will allow selecting a number choice_ui <- function(id) { ns <- NS(id) tagList( @@ -760,7 +760,7 @@ R6 objects, created using the package of the same name, are "traditional" object An R6 object is a data structure that can hold in itself data and functions. Its particularity is that if **it's modified inside a function, this modified value is kept outside the function in which it's called, making it a powerful tool to manage data across the application**. -As this R6 object is not a reactive object and is not meant to be used as such, uncontrolled reactivity of the application is reduced, thus reduces the complexity of handling chain reactions across modules. +As this R6 object is not a reactive object and is not meant to be used as such, uncontrolled reactivity of the application is reduced, thus reducing the complexity of handling chain reactions across modules. Of course, you need to have another special tool in your app to trigger elements. All this will be explained in detail in Chapter \@ref(common-app-caveats) of this book, and you can find an example of this pattern inside the [`{hexmake}`](https://github.com/ColinFay/hexmake/blob/master/R/R6.R) [@R-hexmake] application. @@ -916,7 +916,7 @@ Splitting files using a defined convention is better. Why? Because using a common convention for your files helps the other developers (and potentially you) to know exactly what is contained in a specific file, making it easier to navigate through the codebase, be it for newcomers or for developers already familiar with the software. -As developed in _Refactoring at Scale_ [@lemaire2020], lacking a defined file structure when it comes to the codebase leads to slower productivity in the long run, notably when new engineers join the team: engineers with a knowledge of the file structure have learned how to navigate through the codebase, but new comers will find it hard to understand how everything is organized. +As developed in _Refactoring at Scale_ [@lemaire2020], lacking a defined file structure when it comes to the codebase leads to slower productivity in the long run, notably when new engineers join the team: engineers with a knowledge of the file structure have learned how to navigate through the codebase, but newcomers will find it hard to understand how everything is organized. And of course, in the long run, even developers with a knowledge of the structure can get lost, even more if they haven't worked on the project for months. > Because it's easier to maintain the status quo, instead of proactively beginning to organize related files [...], engineers instead learn to navigate the increasingly sprawling code. diff --git a/04-golem.Rmd b/04-golem.Rmd index d7b86a7..210c734 100644 --- a/04-golem.Rmd +++ b/04-golem.Rmd @@ -74,7 +74,7 @@ Let's focus on the architecture of the default `{golem}` app, and present the ro library(magrittr) ``` -You can create a `{golem}` project, here called `golex`, with RStudio "New project" creation or with command line. +You can create a `{golem}` project, here called `golex`, with RStudio "New project" creation or with the command line. ```{r 04-golem-7, eval=FALSE} golem::create_golem("golex") @@ -143,13 +143,13 @@ The `R/` folder is **the standard folder where you will store all your app funct When you start your project with `{golem}`, this folder is pre-populated with four `.R` files: `app_config.R`, `app_server.R`, `app_ui.R` and `run_app.R`. During the process of building your application, all the core functionalities of your app will be stored in this `R/` directory, which is the standard way to store functions when using the R package framework. -Note that these files are the "core" features of your application itself, and that other .R files also exists. -For example, when you will need to deploy your application on RStudio platforms, `{golem}` will create an `app.R` at the root of your directory.[^golem-1] +Note that these files are the "core" features of your application itself, and that other .R files also exist. +For example, when you need to deploy your application on RStudio platforms, `{golem}` will create an `app.R` at the root of your directory.[^golem-1] The `dev/` folder also contains `.R` scripts, and they are inside this folder as they should not live inside the `R/` folder: they are utilitarian files used during development, not core functionalities of your application. [^golem-1]: `{golem}` will automatically add this file to the `.Rbuildignore` file, i.e. make it be ignored by the package build process. -Inside these `.R` files, contained inside the `R/` folder, you will find the content of your modules (the one added with `golem::add_modules()`) and the utilitarian/business logic functions, built with `golem::add_utils()` and `golem::add_fct()`. +Inside these `.R` files, contained inside the `R/` folder, you will find the content of your modules (the ones added with `golem::add_modules()`) and the utilitarian/business logic functions, built with `golem::add_utils()` and `golem::add_fct()`. Note also that this folder must not contain any sub-folders. @@ -205,7 +205,7 @@ get_golem_config <- function( The `app_config.R` file contains internal mechanics for `{golem}`, notably for referring to values in the `inst/` folder, and to get values from the config file in the `inst/` folder. Keep in mind that if ever you need to change the name of your application, you will need to change it inside the `DESCRIPTION`, but also inside the `app_sys()` function. -To make this process easier, you can use the `golem::set_golem_name()`, which will perform both these actions, plus setting the name inside the config file. +To make this process easier, you can use the `golem::set_golem_name()` function, which will perform both these actions, plus setting the name inside the config file. #### app\_server.R {.unnumbered} @@ -226,7 +226,7 @@ The `app_server.R` file **contains the function for the server logic**. If you are familiar with the classic "ui.R/server.R" approach, this function can be seen as a replacement for the content of the function you have in your `server.R`. Building a complex `{shiny}` application commonly implies using `{shiny}` modules. -If so, you will be adding there a series of `callModule()`, the ones you will get on the very bottom of the file created with `golem::add_module()`. +If so, you will be adding there a series of `callModule()`, the ones you will get at the very bottom of the file created with `golem::add_module()`. You will also find global elements from your server-logic: top-level `reactiveValues()`, connections to databases, setting options, and so forth. @@ -343,10 +343,10 @@ run_app <- function( The `run_app()` function is the one that you will use to launch the app.[^golem-3] -[^golem-3]: Very technically speaking, it is the `print()` from the object outputed by `run_app()` that launches the app, but this is another story. +[^golem-3]: Very technically speaking, it is the `print()` from the object outputted by `run_app()` that launches the app, but this is another story. The body of this function is wrapped inside `with_golem_options()`, which allows you to pass arguments to the `run_app()` function, which can be called later on with `golem::get_golem_options()`. -**The idea here is that you can pass arguments to this function, and that arguments will be later used inside your application to display a specific version of the application**. +**The idea here is that you can pass arguments to this function, and that these arguments will be later used inside your application to display a specific version of the application**. Using this `with_golem_options()` function simplifies the parameterization of `{shiny}` applications, be it during development, when deployed on a server, or when shared as a package. Here are some examples of what you can pass to your shiny application using this pattern: @@ -355,7 +355,7 @@ Here are some examples of what you can pass to your shiny application using this - `run_app(with_mongo = TRUE)` to launch the application with or without a MongoDB back-end (example taken from `{hexmake}`). -- `run_app(dataset = iris)` will make the dataset available with `golem::get_golem_options("dataset")`, so your user can launch the function from their package using a dataset they have created/loaded +- `run_app(dataset = iris)` will make the dataset available with `golem::get_golem_options("dataset")`, so your user can launch the function from their package using a dataset they have created/loaded. ### `golem-config` @@ -392,7 +392,7 @@ For example, in the default example: These options are globally set with: ```{r 04-golem-16, eval = FALSE} -# This functions sets all the default options for your project +# This function sets all the default options for your project set_golem_options() ``` @@ -440,7 +440,7 @@ get_golem_version() You can set these with: ```{r 04-golem-18, eval = FALSE} -# Get the values in the config file +# Set the values in the config file set_golem_name("this") ``` @@ -552,7 +552,7 @@ To do that, you can take several approaches: - Set these values as `run_app()` parameters, but that means that you have to maintain one `app.R` for each server to which you will deploy. - Set everything as environment variables, but that means that you have to do it for every server, and that there is no centralized way to keep track of these variables. -- Set the values in `golem-config.yaml`, and then set a value for the `GOLEM_CONFIG_ACTIVE` environment variable in the environment in which the app is deployed. +- Set the values in `golem-config.yml`, and then set a value for the `GOLEM_CONFIG_ACTIVE` environment variable in the environment in which the app is deployed. This last solution is a convenient one if you want to easily re-deploy your application on various servers without having to (re)set the values for each environment. Note, though, that it shouldn't be used to store sensitive data (for example users and passwords). @@ -594,7 +594,7 @@ knitr::include_graphics("img/GOLEM_CONFIG_ACTIVE.png") The `inst/app/www/` folder contains all files that are made available **at application run time**. Any web application has external files that allow it to run.[^golem-4] -For example, `{shiny}` and its `fluidPage()` function bundles a series of CSS and JavaScript files, notably the `Bootstrap` library, or `jQuery`. These external files enhance your app: CSS for the design part and JavaScript for the interactive part (more or less). +For example, `{shiny}` and its `fluidPage()` function bundle a series of CSS and JavaScript files, notably the `Bootstrap` library, or `jQuery`. These external files enhance your app: CSS for the design part and JavaScript for the interactive part (more or less). On top of that, you can add your own files: your own design with CSS, or your own JavaScript content (as we will see in the last chapters of this book). In order to work, you have to include a link to these files somewhere in the UI. This is what `golem_add_external_resources()` is made for: linking the external resources that you will build with the following functions. @@ -623,7 +623,7 @@ Another common pattern would be: The `dev/` folder is to be used as a **notebook for your development process: you will find here a series of functions that can be used throughout your project**. -The content of these files are specific to `{golem}` here, but the concept of using a script to store all development steps is not restricted to a `{shiny}` application: it could easily be done for any package, and this is something we recommend that you do. +The content of these files is specific to `{golem}` here, but the concept of using a script to store all development steps is not restricted to a `{shiny}` application: it could easily be done for any package, and this is something we recommend that you do. The functions inside these files are the ones used to do some setup, like `usethis::use_mit_license()` or `usethis::use_vignette("my-analysis")`, and add testing infrastructure, like `usethis::use_test("my-function")` or `devtools::check()`. You will also find functions to populate the application like `golem::add_module("my-module")` or `golem::add_js_file("my-script")`. And finally, there are functions you will need once your application is ready: `pkgdown::build_site()`, `rhub::check_for_cran()` or `golem::add_dockerfile()`. diff --git a/05-workflow.Rmd b/05-workflow.Rmd index 528d584..033edaf 100644 --- a/05-workflow.Rmd +++ b/05-workflow.Rmd @@ -1,7 +1,7 @@ # The Workflow {#workflow} Building a robust, production-ready web application will be made easier by following a given workflow. -The one we are advocating is divided in five steps: +The one we are advocating is divided into five steps: + Design + Prototype @@ -9,9 +9,9 @@ The one we are advocating is divided in five steps: + Strengthen + Deploy -In this chapter, we will give an brief overview of the different steps: the rest of the book will cover each of these steps in more depth. +In this chapter, we will give a brief overview of the different steps: the rest of the book will cover each of these steps in more depth. -Of course, as with any workflow, this one is not a one-size-fits-all solution: all projects are unique, with technical requirements, specific planning and team of coder(s). +Of course, as with any workflow, this one is not a one-size-fits-all solution: all projects are unique, with technical requirements, specific planning and a team of coder(s). But we think that following this workflow will help you get good habits when it comes to structuring your application project, even more if you know from day one that the application you are going to work on is a large application, whether in terms of codebase, complexity, or time. Note that the ideas behind this workflow, and its process, could be used outside of a `{shiny}` project: it can be applied to any coding project, even outside of the R world. @@ -35,7 +35,7 @@ For example, the client might write something like "Save the plot inside a datab This first step actually implies a lot of thinking before coding. The main goal of this step is to spend time thinking about the application while you still do not have anything implemented, so that you do not discover blocking elements once it is too late, or at least once you already have written a lot of code. -We have all been in a situation during a project where we tell ourselves: "I wish I had known this sooner": working on designing the application before building it helps lowering the chances for this kind of bad surprise. +We have all been in a situation during a project where we tell ourselves: "I wish I had known this sooner": working on designing the application before building it helps lower the chances for this kind of bad surprise. This first part of the workflow will span three chapters: @@ -46,7 +46,7 @@ These topics are vast topics, and a lot of literature and online resources exist + Chapter \@ref(dont-rush-into-coding), "Don't rush into coding" underlines why "coding first" might not be the best strategy when it comes to building a production application. We will also quickly introduce concept maps, and list some of the common questions you might want to ask the people involved in the project. -+ Finally, this first part of the workflow covers a gentle introduction to CSS, which might be a crucial skill to master when it comes to sending an application to production: either your clients already have a CSS template that they want to include in the application, or they want their application to have the color and design that match the one from the company. ++ Finally, this first part of the workflow covers a gentle introduction to CSS, which might be a crucial skill to master when it comes to sending an application to production: either your clients already have a CSS template that they want to include in the application, or they want their application to have the color and design that match the one from the company. Also, when building a professional application, chances are that you will want your app to stand out from the crowd: hence a little bit of CSS. This part is included in the design part because it is something that you might want to think about from the very beginning: for example, some companies have pre-existing `{shiny}` templates, they might want to include specific fonts, logo, icons, etc. These are things better known before starting to code: it is easier to start working inside a `{shiny}` template than migrating an existing code to a template. @@ -66,7 +66,7 @@ On the other hand, you (or someone from your team), will be working on building For this point, you can use what we call a "Rmd-first" approach, by combining R functions with the writing of vignettes that describe the internals of the application. This part of the workflow will be developed in two chapters: -+ Chapter (\@ref(setting-up-for-success)), "Setting up for success with `{golem}`", will cover the basics of getting started with the `{golem}` package so that you can start your prototyped application with solid foundation. ++ Chapter (\@ref(setting-up-for-success)), "Setting up for success with `{golem}`", will cover the basics of getting started with the `{golem}` package so that you can start your prototyped application with a solid foundation. + Chapter (\@ref(building-ispum-app)), "Building an “ipsum-app”" will cover the importance of prototyping when it comes to building applications, then present `{shinipsum}` and `{fakir}`, and finally will introduce how you can use the "Rmd First" methodology to prototype your application back-end. @@ -75,7 +75,7 @@ This part of the workflow will be developed in two chapters: The __build__ part is the one where you will combine the business (or back-end) logic with the front-end. In this third part, you will work on the core engine of the application, making the business logic work inside the interactive logic of your application. -This step of the workflow is covered in _Building app with `{golem}`_ (\@ref(build-app-golem)), a chapter that presents the various functions you can use to build your application, _i.e_ the one you will be using to combine your back-end and front-end. +This step of the workflow is covered in _Building the App with `{golem}`_ (\@ref(build-app-golem)), a chapter that presents the various functions you can use to build your application, _i.e_ the ones you will be using to combine your back-end and front-end. In this step, we will cover: @@ -90,13 +90,13 @@ In this step, we will cover: The __strengthen__ part covers how to ensure your application is immortal, in the sense that we defined in Chapter \@ref(successful-shiny-app) of this book. In this part, we will go through unit tests, reproducible development environments, version control, and continuous integration in the context of `{shiny}` applications. -Building a solid testing suite is crucial to the success of a project, as it allows a project to be stable in the long run, be it when you will want to add new feature or refactor existing code: +Building a solid testing suite is crucial to the success of a project, as it allows a project to be stable in the long run, be it when you will want to add new features or refactor existing code: > Refactoring requires we be able to confidently ensure that behavior remains identical at every iteration. We can increase our confidence that nothing has changed by writing a suite of tests (unit, integration, end-to-end), and we should not seriously consider moving forward with any refactoring effort until we’ve established sufficient test coverage. > > _Refactoring at Scale_ [@lemaire2020] -This step of the workflow will span over chapters. +This step of the workflow will span over two chapters. + The first one, "Build yourself a safety net" (Chapter \@ref(build-yourself-safety-net)), details how to build a testing environment for your `{shiny}` application, be it for testing the back-end or the front-end. In this chapter, you will be introduced to `{testthat}` for testing your application back-end, tools that are more linked to testing the front-end like NodeJS `puppeteer` module, `{shinytest}` and `{crrry}` for testing interactive logic, `{shinyloadtest}` and `{dockerstats}` for testing your application load. @@ -120,7 +120,7 @@ These questions (and more) will be covered in the __deploy__ part of this book. In this part, we will present a series of methods to prepare your application to be deployed on various environments, notably: -+ Sharing your application as a package so that it can be installed manually, through GitHub, or shared on a package repository like the CRAN or BioConductor ++ Sharing your application as a package so that it can be installed manually, through GitHub, or shared on a package repository like CRAN or BioConductor + Sending it to an RStudio platform + Building a Docker image to serve your app on a cloud provider diff --git a/06-ux-matters.Rmd b/06-ux-matters.Rmd index 29a8af7..a61c720 100644 --- a/06-ux-matters.Rmd +++ b/06-ux-matters.Rmd @@ -27,7 +27,7 @@ There are mainly two contexts where you will be building a web app with R: for p But both cases have something in common: people will want the app to be usable, **easily** usable. If people use your app in a professional context, they do not want to fight with your interface, read complex manuals, or lose time understanding what they are supposed to do and how they are supposed to use your application, at least when it comes to the core usage of the application. -This core usage needs to be "self-explanatory", in the sense that, **if possible, the main usage of the application does not require reading the manual**; On the other hand, more advanced/rarely used features will need more detailed documentation. +This core usage needs to be "self-explanatory", in the sense that, **if possible, the main usage of the application does not require reading the manual**. On the other hand, more advanced/rarely used features will need more detailed documentation. In other words, they want an efficient tool, something that - beyond being accurate - is easy to grasp. In a professional context, when it comes to "business applications", remember that the quicker you understand the interface, the better the user experience. @@ -44,9 +44,9 @@ These two rules aim at solving one issue: the bigger the cognitive load of your One big lie we tell ourselves as developers is that the end user will use the app the way we designed it to be used (though to be honest, this is not true for any software). We love to think that when faced with our app, the users will carefully read the instructions and make a rational decision based on careful examination of the inputs before doing what we expect them to do. -But the harsh truth is, that it is not what happens. +But the harsh truth is that it is not what happens. -First of all, users rarely carefully read all the instructions: they **scan** and perform the first action that more or less matches what they need to do, i.e., they **satisfice** (a portmanteau of satisfy and suffice); a process shown in Figure \@ref(fig:06-ux-matters-1). +First of all, users rarely carefully read all the instructions: they **scan** and perform the first action that more or less matches what they need to do, i.e., they **satisfice** (a portmanteau of satisfy and suffice), a process shown in Figure \@ref(fig:06-ux-matters-1). Navigating the web, users try to optimize their decision, not by making the decision that would be "optimal", but by doing the first action that is sufficiently satisfactory in relevance. They behave like that for a lot of reasons, but notably because they want to be as quick as possible on the web, and because the cost of being wrong is very low most of the time - even if you make the wrong decision on a website, chances are that you are just a "return" or "cancel" button away from canceling your last action. @@ -80,7 +80,7 @@ For example, most of the time we will first change the package name or upload an Once users have scanned the page, they perform the first action that seems reasonable, or as coined in "Rational Choice and the Structure of the Environment" by Herbert A. Simon, "**organisms adapt well enough to 'satisfice'; they do not, in general, optimize."**. In other words, **"As soon as we find a link that seems like it might lead to what we're looking for, there's a very good chance that we'll click it"** (_Don't Make Me Think_, [@stevekrug2014]). -What that also means is that user might perform what you'd expect to be "irrational" choices. +What that also means is that users might perform what you'd expect to be "irrational" choices. As they are scanning your application, they might do something unexpected, or use a part of your app in a way that you would not expect it to be used. For example, if you are creating an app that is designed to take input data that has to be filled in following a specific form, you **need** to check that this requirement is fulfilled, or you will end up debugging errors on uncommon entries. @@ -142,11 +142,11 @@ In fact, this is a crucial thing when it comes to making your app successful: ** That means that even when your R code fails, the whole app should not fail. If the R code fails for some reason, the user should either get nothing back or an informative bug message, not be faced with a grayish version of the application.[^ux-matters-3] Note that using external widgets, like the one from the `{DT}` package (or any other that binds to an external JavaScript library), can make this principle harder to apply: as you have less control over what is happening when using this widget, gracefully handling errors can be tricky. -Indeed, `{DT}` sometimes returns errors that originates from the user's browser, so that has nothing to do with R. +Indeed, `{DT}` sometimes returns errors that originate from the user's browser, so that has nothing to do with R. In that case, it might be hard to catch this error and gracefully manage it. The only upside of this error is that it does not crash the whole application. -[^ux-matters-3]: If you want something different from this grayish screen when `{shiny}` fails, you can have a look at the `{sever}` package [@R-sever], which allows to implement custom disconnected screen and error messages. +[^ux-matters-3]: If you want something different from this grayish screen when `{shiny}` fails, you can have a look at the `{sever}` package [@R-sever], which allows you to implement custom disconnected screens and error messages. Because of the way `{shiny}` is designed, a lot of R errors will make the `{shiny}` app fail completely. If you have not thought about this upfront, that means that a user might use the app for 10 minutes, do a series of specifications, enter parameters and data, only for the app to completely crash at some point. @@ -223,7 +223,7 @@ This rule is also known as "Principle of Least Astonishment." > _The Art of UNIX Programming_ [@ericraymond2003] When we are browsing the web, **we have a series of pre-conceptions about what things are and what they do**. -For example, we expect an underline text to be clickable, so there is a good chance that if you use underlined text in your app, the user will try to click on it. +For example, we expect an underlined text to be clickable, so there is a good chance that if you use underlined text in your app, the user will try to click on it. Usually, the link is also colored differently from the rest of the text. The same goes for the pointer of the mouse, which usually switches from an arrow to a small hand with a finger up. A lot of other conventions exist on the web, and you should endeavor to follow them: a clickable link should have at least one of the properties we just described—and if it is neither underlined nor colored and does not change the pointer when it is hovered, chances are that the user will not click on it. @@ -243,7 +243,7 @@ Of course, this is not an absolute rule, and there is always room for creativity Weirdly enough, that is an easy thing to spot when we arrive on a web page or an app: it can either feel "natural", or you can immediately see that something is off. The hard thing is that it is something you spot when you are a new-comer: developing the app makes us so familiar with the app that we might miss when something is not used the way it is conventionally used.[^ux-matters-4] -[^ux-matters-4]: For a good summary of these, see "The cranky user: The Principle of Least Astonishmen" +[^ux-matters-4]: For a good summary of these, see "The cranky user: The Principle of Least Astonishment" Let's exemplify this with the "Render" button from the [`{tidytuesday201942}`](https://connect.thinkr.fr/tidytuesday201942/) application. This app is built on top of Bootstrap 4, which has no CSS class for a `{shiny}` action button.[^ux-matters-5] @@ -285,7 +285,7 @@ A good and simple way to do that is to hide elements at step n+1 until all the r Indeed, you can be sure that if step 2 relies on step 1 and you did not hide step 2 until you have everything you need, users will go to step 2 too soon. Another way to help this readability is to ensure some kind of linear logic through the app: step 1, data upload, step 2, data cleaning, step 3, data visualization, step 4, exporting the report. -And organized your application around this logic, from left to right / right to left, or from top to bottom. +And organize your application around this logic, from left to right / right to left, or from top to bottom. Let's compare `{tidytuesday201942}` to `{hexmake}`—one has a clear progression, `{hexmake}`, and has been designed as such: the upper menus design the stickers, and then once they are filled you can download them. There is a progression here, from top to bottom. @@ -305,12 +305,12 @@ That way, you can help the user navigate through the app, by reducing the cognit And do this for all the elements in your app: for example, with `{hexmake}`, we start with filled fields and a hex sticker which is ready, so that even if you start with the download part, the application would still work. If we had chosen another pattern, such as making the user fill in everything before being able to download, we would have needed to make downloading impossible until all fields are filled. -Another example from this application is the use of a MongoDB back-end to store the hex stickers: if the application is launched with `with_mongo` set to FALSE, the user will not see any buttons or field that refers to this option. +Another example from this application is the use of a MongoDB back-end to store the hex stickers: if the application is launched with `with_mongo` set to FALSE, the user will not see any buttons or fields that refer to this option. Think about all the times when you are ordering something on the internet, and need to fill specific fields before being able to click on the "Validate" button. Well, apply that approach to your app; that will prevent unwanted mistakes. -Note that when using the `golem::use_utils_ui()` function, you will end with a script of UI tools, one being `with_red_star`, which adds a little red star at the end of the text you are entering, a common pattern for signifying that a field is mandatory: +Note that when using the `golem::use_utils_ui()` function, you will end up with a script of UI tools, one being `with_red_star`, which adds a little red star at the end of the text you are entering, a common pattern for signifying that a field is mandatory: ```{r 06-ux-matters-7, echo = FALSE} with_red_star <- function(text) { @@ -364,7 +364,7 @@ server <- function( ){ output$plt <- renderPlot({ # If the length of the input is 0 - # (i.e. nothing is selected),we show + # (i.e. nothing is selected), we show # a feedback to the user in the form of a text # If the length > 0, we remove the feedback. if (length(input$species) == 0){ @@ -375,7 +375,7 @@ server <- function( } else { hideFeedback("species") } - # req() allows to stop further code execution + # req() allows us to stop further code execution # if the condition is not a truthy. # Hence if input$species is NULL, the computation # will be stopped here. @@ -430,8 +430,8 @@ But let's focus on a smaller scope, and think about some things that can be thou When designing an app, you will be designing the way users will navigate through the app. And most of the time, we design with the idea that the users will perform a "correct selection" pattern. -Something like: "The user will select 40 on the `sliderInput()` and the plot will update automatically. Then the user will select the element they need in the `selectInput()` and the plot will update automatically*". -When in reality what will happen is: "*The user will click on the slider, aim at 40 but will reach 45, then 37, then 42, before having the right amount of 40. Then they will select something in the `selectInput()`, but chances are, not the correct one from the first time." +Something like: "*The user will select 40 on the `sliderInput()` and the plot will update automatically. Then the user will select the element they need in the `selectInput()` and the plot will update automatically*". +When in reality what will happen is: "*The user will click on the slider, aim at 40 but will reach 45, then 37, then 42, before having the right amount of 40. Then they will select something in the `selectInput()`, but chances are, not the correct one from the first time.*" In real-life usage, **people make mistakes while using the app** (and even more when discovering the application): they do not move the sliders to the right place, so if the application reacts to all of the moves, the experience using the app can be bad: in the example above, full reactivity means that you will get 4 "wrong" computations of the plot before getting it right. @@ -500,7 +500,7 @@ When building professional `{shiny}` applications, you have to keep in mind that **A large audience means that there is a chance that your app will be used by people with visual, mobility, or maybe cognitive disabilities**.[^ux-matters-7] Web accessibility deals with the process of making the web available to people with disabilities. -[^ux-matters-7]: And of course, other type of disabilities. +[^ux-matters-7]: And of course, other types of disabilities. > The Web is fundamentally designed to work for all people, whatever their hardware, software, language, location, or ability. > When the Web meets this goal, it is accessible to people with a diverse range of hearing, movement, sight, and cognitive ability. @@ -573,17 +573,17 @@ shinyApp(ui, server) ``` What makes these two things similar (semantic tags and tag metadata) is that they are both unseen by users without any impairment: if the image is correctly rendered and the user is capable of reading images, chances are that this user will see the image. -But these elements are made for people with disabilities, and especially users who might be using screen-to-speech technologies: these visitors use a software that scans the textual content of the page and reads it, and that helps navigate through the page. +But these elements are made for people with disabilities, and especially users who might be using screen-to-speech technologies: these visitors use software that scans the textual content of the page and reads it, and that helps navigate through the page. This navigation is also crucial when it comes to screen-to-speech technology: such software will be able to read the `` tag, jump to the `<nav>`, or straight to the `<article>` on the page. Hence the importance of structuring the page: these technologies need the app to be built in a structured way, so that it is possible to jump from one section to another, and other common tasks a fully capable user will commonly do. -Some other tags exist and can be used for semantic purpose: for example, `<address>`, `<video>`, or `<label>`. +Some other tags exist and can be used for semantic purposes: for example, `<address>`, `<video>`, or `<label>`. #### C. Navigation {.unnumbered} Your app user might also have mobility impairment. -For example, some with Parkinson's disease might be using your app, or someone with a handicap making it harder for them to move their hand and click. +For example, someone with Parkinson's disease might be using your app, or someone with a handicap making it harder for them to move their hand and click. For these users, moving an arm to grab the mouse might be challenging, and they might be navigating the web using their keyboard only. When building your app, thinking about how these users will be able to use it is crucial: maybe there are so many buttons to which they need to **move their mouse and eventually click** that they will not be able to use it. @@ -635,7 +635,7 @@ Keeping in mind this prevalence of color blindness is even more important in the If designed wrong, dataviz can be unreadable for some specific type of color blindness. That is why we recommend using the `viridis` [@R-viridis] palette, which has been created to be readable by the most common types of color blindness. -Here are, for example, a visualization through the lens of various typed of color blindness: +Here is, for example, a visualization through the lens of various types of color blindness: ```{r 06-ux-matters-15 } # This function generates a plot for an diff --git a/07-step-by-step-design.Rmd b/07-step-by-step-design.Rmd index 7265bdc..0833d46 100644 --- a/07-step-by-step-design.Rmd +++ b/07-step-by-step-design.Rmd @@ -20,7 +20,7 @@ Yes, we all have been in this situation: realizing too late that the thing we ha And what about "Oh I wish I had realized sooner that this package existed before trying to implement my own functions to do that!"[^step-by-step-design-1] Same thing: we're jumping straight into solving a programming problem when someone else has open-sourced a solution to this very same problem. -[^step-by-step-design-1]: Given the dynamic of the R community, there is no way to completely avoid this: new packages are created and publish every day, so there is no way to be aware of everything. +[^step-by-step-design-1]: Given the dynamic of the R community, there is no way to completely avoid this: new packages are created and published every day, so there is no way to be aware of everything. But trying to assess what exists before jumping into coding will definitely save you some time in the long run. Of course, implementing your own solution might be a good thing in specific cases: avoiding heavy dependencies, incompatible licensing, the joy of the intellectual challenge, but **when building production software, it is safer to go for an existing solution if there is one and it fits in the project: existing packages/software that are widely used by the community and by the industry benefit from wider testing, wider documentation, and a larger audience if you need to ask questions**. @@ -28,7 +28,7 @@ And of course, it saves time, be it immediately or in the long run: re-using an [^step-by-step-design-2]: Of course, it is not an absolute rule: you might also inherit from the bug created by the open source solution. -Note also that assessing that a dependency/technology is a good choice for an application is not an easy task: there is a difference between *thinking* something will be the good choice and *knowing* that this choice is the correct one. +Note also that assessing that a dependency/technology is a good choice for an application is not an easy task: there is a difference between *thinking* something will be the right choice and *knowing* that this choice is the correct one. Most of the time, when faced with a new technology, it makes sense to take some time to write a small prototype that tests the features we want to use. This process of prototyping small applications to test features is made easier notably by using the `{shinipsum}` package, which we will see in Chapter \@ref(building-ispum-app). @@ -50,11 +50,11 @@ Here is a non-exhaustive list of places you can look if you are stuck/looking fo #### Web {.unnumbered} -- [Mozilla developer center](https://developer.mozilla.org/) is one of the most comprehensive resource platforms when it comes to web technologies (HTML, CSS, and JavaScript) +- [Mozilla developer center](https://developer.mozilla.org/) is one of the most comprehensive resource platforms when it comes to web technologies (HTML, CSS, and JavaScript). - [Google Developer Center](https://developers.google.com/) also has a series of resources that can be helpful when it comes to web technologies. - [FreeCodeCamp](https://www.freecodecamp.org/) contains more than 2000 hours of free courses about web technologies, plus a blog and forum. -### About concept map +### About concept maps Using a concept map to think about your app can be a valuable method to help you grasp the big picture of your application. @@ -75,7 +75,7 @@ As you can see, we are not detailing the technical implementations: we are not w The goal of a concept map is to think about the big picture, to see the "who and what" of the application. Here, creating this concept map helps us list the flow of the app: there is a user that wants to configure a hex, built with a default image or with an uploaded one, and once this hex is finished, the user can either download it or register it in a database. This database can be browsed and restore hex. -The user can also export a `.hex` file, that can restore an app configuration. +The user can also export a `.hex` file, which can restore an app configuration. Once this general flow is written down, you can get back to it several times during the process of building the app, but it is also a perfect tool at the end to see if everything is in place: once the application is finished, we can question it: @@ -115,7 +115,7 @@ And developing for mobiles requires a different kind of mindset.[^step-by-step-d [^step-by-step-design-3]: For developing an app that is mobile first, you can have a look at the great `{shinyMobile}` [@R-shinyMobile] package made by the amazing Rinterface (<https://rinterface.com/>) team. -Another good reason why talking to the users is an important step, is that most of the time, **people writing specifications are not the end users and will either request too many features or not enough**. +Another good reason why talking to the users is an important step is that most of the time, **people writing specifications are not the end users and will either request too many features or not enough**. Do the users really need that many interactive plots? Do they actually need that much granularity in the information? Will they really see a `datatable` of 15k lines? @@ -130,7 +130,7 @@ On top of that, remember all these things we saw in the last chapter about acces ### Building personas The persona is a concept borrowed from design and marketing that refers to fictional characters that will serve as a user type. -In other words**, a persona is a character that represents the "typical" behavior and traits for a group of users that will interact with your product**. +In other words, **a persona is a character that represents the "typical" behavior and traits for a group of users that will interact with your product**. > A persona consists of a description of a fictional person who represents an important customer or user group for the product, and typically presents information about demographics, behavior, product usage, and product-related goals, tasks, attitudes, etc. > @@ -142,7 +142,7 @@ Will they understand it? Do we need to add extra information? Will they find this useful? -Asking these kinds of questions helps you take a step back from feature implementation and re-focus on what matters: we are building application for someone else, who will eventually use it. +Asking these kinds of questions helps you take a step back from feature implementation and re-focus on what matters: we are building an application for someone else, who will eventually use it. > The benefits of personas are that they enable designers to envision the end user's needs and wants, remind designers that their own needs are not necessarily the end users' needs, and provide an effective communication tool, which facilitates better design decisions.\ > @@ -158,7 +158,7 @@ And don't hesitate to detail these fictional characters as "[p]ersonas are consi From time to time, you are building a `{shiny}` app on top of an existing code-base: either scripts with business logic, a package if you are lucky, or a PoC for a `{shiny}` app. -These kinds of projects are often referred to as "brownfield projects", in opposition to "greenfield projects", borrowing the terminology from urban planning: **a greenfield project being one where you are building on "evergreen" lands, while a brownfield project is building on lands that were, for example, industrial lands, and which will need to be sanitized, as they potentially contain waste or pollution, constructions need to be destroyed, roads needs to be deviated, and all these things that can make the urban planning process more complex**. +These kinds of projects are often referred to as "brownfield projects", in opposition to "greenfield projects", borrowing the terminology from urban planning: **a greenfield project being one where you are building on "evergreen" lands, while a brownfield project is building on lands that were, for example, industrial lands, and which will need to be sanitized, as they potentially contain waste or pollution, constructions need to be destroyed, roads need to be deviated, and all these things that can make the urban planning process more complex**. Then, you can extend this to software engineering, where a greenfield project is the one that you start from scratch, and a brownfield project is one where you need to build on top of an existing code-base, implying that you will need to do some extra work before actually working on the project. \newpage @@ -167,8 +167,8 @@ Then, you can extend this to software engineering, where a greenfield project is > > _The DevOps Handbook_ [@genekim2016] -Depending on how you chose to handle it, starting from a codebase that is already written can either be very much helping, or you can be shooting yourself in the foot. -Most of the time, `{shiny}` projects are not built as reproducible infrastructures: you will find a series of `library()` calls, no functions structure *per se*, no documentation, and no tests. +Depending on how you chose to handle it, starting from a codebase that is already written can either be very helpful, or you can be shooting yourself in the foot. +Most of the time, `{shiny}` projects are not built as reproducible infrastructures: you will find a series of `library()` calls, no function structure *per se*, no documentation, and no tests. In that case, we would advise you to do it "the hard way", or at least what seems to be the hard way: throw the app away and start from scratch. Well, not really from scratch: **extract the core business logic of the app and make it a package**. @@ -186,11 +186,11 @@ That's why it is better to split the business and app logic from the very beginn ### Deployment There are so many considerations about deployment that it will be very hard to list them all, but keep in mind that **if you do not ask questions about where your application will be deployed from the very beginning, sending it to production might become a painful experience**. -Of course, it is more or less solved if you are deploying with Docker: if it works in a container on your machine, it should work in production, but it is not as simple as that: for example, building a `{shiny}` application that will be used by 10 people is not the same as building an application that needs to scale to 50.000 users. +Of course, it is more or less solved if you are deploying with Docker: if it works in a container on your machine, it should work in production, but it is not as simple as that: for example, building a `{shiny}` application that will be used by 10 people is not the same as building an application that needs to scale to 50,000 users. Learning at the end of the project that "now we need to scale to a very large user base" might prevent the deployment from being successful, as this kind of scale implies specific consideration while building. But that is just the tip of the iceberg of things that can happen. -Let's stop for a little story: once upon a time, a team of developers was missioned to build an app, and one feature of the app was to do some API requests. +Let's stop for a little story: once upon a time, a team of developers was commissioned to build an app, and one feature of the app was to do some API requests. So far so good, nothing too complicated, until they discovered that the server where the app was going to be deployed does not have access to the internet, making it impossible to issue API requests from the server. Here, the containers worked on the dev machines, as they had access to the internet. Once deployed, the app stopped working, and the team lost a couple of days of exchanges with the client, trying to debug the API calls, until we realized that the issue was not with the app, but with the production server itself: and nobody in the team, not the developers or the client, thought about asking about internet access for the server. diff --git a/08-step-by-step-prototype.Rmd b/08-step-by-step-prototype.Rmd index 311e284..ea9be93 100644 --- a/08-step-by-step-prototype.Rmd +++ b/08-step-by-step-prototype.Rmd @@ -5,7 +5,7 @@ Before starting to prototype and build anything, initialize a `{golem}` [@R-golem] project! This will help you start your application on solid ground, and once the project is ready to be filled, you can start prototyping right inside it. -The general workflow for "prototype and build" is the following: the project manager sets up a `{golem}` project, where the first steps are filled, the general structure (potentially with `{shiny}` module) is set, and then the project is registered to the version control system. +The general workflow for "prototype and build" is the following: the project manager sets up a `{golem}` project, where the first steps are filled, the general structure (potentially with `{shiny}` modules) is set, and then the project is registered to the version control system. Once we have this structure, package and modules combined, we can start prototyping the UI inside the module, work on the CSS and JavaScript elements that might be needed, and the back-end functionalities inside Rmarkdown files. And then, once these two prototyping sides are finished, we work on the integration of everything inside the reactive context. @@ -14,7 +14,7 @@ In this chapter and in chapter 11, we will be presenting the `{golem}` package i ## Create a `{golem}` -Once `{golem}` is installed and available on your computer, you can go to File \> New Project... in RStudio, and choose "Package for `{shiny}` app Using golem" input. +Once `{golem}` is installed and available on your computer, you can go to File \> New Project... in RStudio, and choose the "Package for `{shiny}` app Using golem" input. If you want to do it through the command line, you can use: @@ -118,12 +118,12 @@ golem::fill_desc( Then, call the `golem::set_golem_options()` function, which will add information to the `golem-config.yml` file, and set the `{here}` [@R-here] package root sentinel. `{here}` is an R package designed to handle directory management in R. -When used in combination with `{golem}`, `{here}` helps ensure that everything you do in your console is performed relatively to the root directory of your project: the one containing the `DESCRIPTION` of your application. +When used in combination with `{golem}`, `{here}` helps ensure that everything you do in your console is performed relative to the root directory of your project: the one containing the `DESCRIPTION` of your application. That way, even if you change the working directory of your R session to a subfolder, you will still be able to create modules and CSS files in the correct folder. ### Set common files -If you want to use the MIT license, add README, a code of conduct, a lifecycle badge, and NEWS. +If you want to use the MIT license, add a README, a code of conduct, a lifecycle badge, and NEWS. ```{r 08-step-by-step-prototype-5, eval = FALSE} # You can set another license here @@ -144,7 +144,7 @@ usethis::use_git() ### Use recommended elements -`golem::use_recommended_tests()` and `golem::use_recommended_deps()` sets a default testing infrastructure and adds dependencies to the application. +`golem::use_recommended_tests()` and `golem::use_recommended_deps()` set a default testing infrastructure and add dependencies to the application. ### Add utility functions @@ -159,7 +159,7 @@ golem::use_utils_ui() golem::use_utils_server() ``` -In this file, you will, for example, find `list_to_li()`, which is a function to turn an R list into an HTML list or `with_red_star()`, a function to add a small red star after a UI input, useful for communicating that an input is mandatory. +In this file, you will, for example, find `list_to_li()`, which is a function to turn an R list into an HTML list, or `with_red_star()`, a function to add a small red star after a UI input, useful for communicating that an input is mandatory. ### Changing the favicon diff --git a/09-prototyping.Rmd b/09-prototyping.Rmd index 8efd225..5890572 100644 --- a/09-prototyping.Rmd +++ b/09-prototyping.Rmd @@ -11,7 +11,7 @@ And yet another rule from _The Art of Unix Programming_: "Rule of Optimization: Prototype before polishing. **Get it working before you optimize it**." Getting things to work before trying to optimize the app is always a good approach: -- **Making things work before working on low-level optimization makes the whole engineering process easier**: having a "minimal viable product" that works, even if slowly and not perfectly, gives a stronger sense of success to the project. For example if you are building a vehicle, it feels more of a success to start with a skateboard than with a wheel: you quickly have a product that can be used to move, not waiting for the end of the project before finally having something useful. Building a skateboard helps the developer maintain a sense of accomplishment throughout the life of the project: the quicker you can have a running program, a MVP (Minimum Viable Product, as seen on Figure \@ref(fig:09-prototyping-1)), the better. +- **Making things work before working on low-level optimization makes the whole engineering process easier**: having a "minimal viable product" that works, even if slowly and not perfectly, gives a stronger sense of success to the project. For example if you are building a vehicle, it feels more of a success to start with a skateboard than with a wheel: you quickly have a product that can be used to move, not waiting for the end of the project before finally having something useful. Building a skateboard helps the developer maintain a sense of accomplishment throughout the life of the project: the quicker you can have a running program, an MVP (Minimum Viable Product, as seen on Figure \@ref(fig:09-prototyping-1)), the better. \newpage @@ -185,7 +185,7 @@ and text, image, `ggplotly`, `dygraph`, and `DT`. `{shinipsum}` is also a good tool if you want to demonstrate what a given UI framework will look like if used in `{shiny}`. This is, for example, what you find with `{golemhtmltemplate}`, available at [engineering-shiny.org/golemhtmltemplate/](https://engineering-shiny.org/golemhtmltemplate/), which uses a W3 web page template.[^prototyping-2] -[^prototyping-2]: This application is also a demonstration of how to build a `{golem}` application using `htmltemplate()`. +[^prototyping-2]: This application is also a demonstration of how to build a `{golem}` application using `htmlTemplate()`. ### Using `{fakir}` for fake data generation diff --git a/10-step-by-step-build.Rmd b/10-step-by-step-build.Rmd index ee89c5a..d0ff1fb 100644 --- a/10-step-by-step-build.Rmd +++ b/10-step-by-step-build.Rmd @@ -33,7 +33,7 @@ But what about other dependencies like `{ggplot2}`? These ones need to be added [^step-by-step-build-1]: The idea with this function is to provide a shortcut for adding commonly used dependencies, so that you don't have to do it by hand. -Here is how to process for a new dependency: +Here is how to proceed for a new dependency: - Open the `dev/02_dev.R` script. - Call the `use_package()` function from `{usethis}`: `usethis::use_package("pkg.you.want.to.add")`. @@ -53,11 +53,11 @@ There are two places where the dependencies of your application need to be manag - The `NAMESPACE` file describes how your app interacts with the R session at run time, i.e. **when your application is launched**. With this `NAMESPACE` file, you can specify only a subset of functions to import from other packages: for example, you can choose to import only `renderDT()` and `DTOutput()` from `{DT}`, instead of importing all the functions. This selective import mechanism allows you to avoid namespace conflicts: for example, between `jsonlite::flatten()` and `purrr::flatten()`.[^step-by-step-build-4] - To do so, we will need to go to every script that defines one or several function/s, and add a `{roxygen2}` [@R-roxygen2] tag, in the following form : `#' @importFrom jsonlite fromJSON` and `#' @importFrom purrr flatten`: that way, you are only importing `fromJSON()` from `{jsonlite}`. + To do so, we will need to go to every script that defines one or several function/s, and add a `{roxygen2}` [@R-roxygen2] tag, in the following form: `#' @importFrom jsonlite fromJSON` and `#' @importFrom purrr flatten`: that way, you are only importing `fromJSON()` from `{jsonlite}`. [^step-by-step-build-3]: Note that most of the time, you will not be filling this by hand, but by using `usethis::use_package()`. -[^step-by-step-build-4]: This can be pretty common as `{jsonlite}` might import `JSON` files as list, and `{purrr}` has pretty powerful tools for manipulating lists. +[^step-by-step-build-4]: This can be pretty common as `{jsonlite}` might import `JSON` files as lists, and `{purrr}` has pretty powerful tools for manipulating lists. Note that you can also use explicit namespacing, i.e. the `pkg::function()` notation inside your code. And if you need a little help to identify dependencies, all the explicitly namespaced calls (`pkg::function()`) can be scraped using the `{attachment}` [@R-attachment] package: @@ -86,7 +86,7 @@ To learn more about the details of how to manage dependencies, and about the `DE ## Submodules and utility functions When building a large application, you **will be splitting your codebase into smaller pieces**. -In Chapter \@ref(structuring-project), "Structuring Your Project", that these utilitarian functions should be defined in files that are prefixed with a specific term. +In Chapter \@ref(structuring-project), "Structuring Your Project", we saw that these utilitarian functions should be defined in files that are prefixed with a specific term. In the `{golem}` world, these are `utils_*` and `fct_*` files: - `utils_*` files contain small functions that might be used several times in the application. @@ -100,8 +100,8 @@ golem::add_fct( "helpers" ) golem::add_utils( "helpers" ) ``` -- The first will create a `R/fct_helpers.R` file. -- The second will create a `R/utils_helpers.R` file. +- The first will create an `R/fct_helpers.R` file. +- The second will create an `R/utils_helpers.R` file. The idea, as explained before, is that as soon as you open a `{golem}`-based project, you are able to identify what the files contain, without having to open them.[^step-by-step-build-5] @@ -112,7 +112,7 @@ For example, the `{hexmake}` app has two of these files, [`R/utils_ui.R`](https: The `fct_*` files are to be used with larger functions, which are more central to the application, but that might not fit into a specific module. For example, in `{hexmake}`, you will find [`R/fct_mongo.R`](https://github.com/ColinFay/hexmake/blob/master/R/fct_mongo.R), which is used to handle all the things related to connecting and interacting with the Mongodb database. -As you can see, the difference is that `fct_*` file are more "topic centered", in the sense that they gather functions that relate to a specific feature of the application (here, the database), while `utils_*` files are more used as a place to put miscellaneous functions. +As you can see, the difference is that `fct_*` files are more "topic centered", in the sense that they gather functions that relate to a specific feature of the application (here, the database), while `utils_*` files are more used as a place to put miscellaneous functions. Note that when building a module with `golem::add_module()`, you can add a module-specific `fct_*` or `utils_*` file: @@ -200,7 +200,7 @@ Note that you can also perform code coverage locally, using the `{covr}` [@R-cov code_coverage <- covr::package_coverage() ``` -For example, Figure \@ref(fig:10-step-by-step-build-11) is the output of running the `package_coverage()` function on the `{golem}` package on the 2020-04-29 on the `dev` branch: +For example, Figure \@ref(fig:10-step-by-step-build-11) is the output of running the `package_coverage()` function on the `{golem}` package on 2020-04-29 on the `dev` branch: (ref:golemcodecoverageresults) {golem} code coverage results. @@ -240,10 +240,10 @@ Note also that if you want to add the code coverage of your application inside a #### B. Continuous Integration {.unnumbered} Continuous integration, on the other hand, is ensuring the software is still working whenever a change is made by one of the developers. -The idea is to add to the centralized version control system (for example, Git)[^step-by-step-build-6] a service like Travis CI, GitHub Action (if you are on GitHub), or GitLab CI (for GitLab) that runs a series of commands whenever something is integrated in the repository, i.e. every time a change to the codebase is made. +The idea is to add to the centralized version control system (for example, Git)[^step-by-step-build-6] a service like Travis CI, GitHub Actions (if you are on GitHub), or GitLab CI (for GitLab) that runs a series of commands whenever something is integrated in the repository, i.e. every time a change to the codebase is made. In other words, every time a new piece of code is sent to the central repository, a service runs regression tests that check that the software is still in a valid, working state. -[^step-by-step-build-6]: We will get back to version control in the Chapter \@ref(version-control), "Version Control".. +[^step-by-step-build-6]: We will get back to version control in Chapter \@ref(version-control), "Version Control". You can set up various continuous integration services automatically by using functions from the `{usethis}` package: @@ -289,7 +289,7 @@ golem::cat_dev("In dev\n") Of course, chances are you do not only need to print things, you might want to use other functions. Good news! -You can make any function being "dev-dependent" with the `make_dev()` function: +You can make any function "dev-dependent" with the `make_dev()` function: ```{r 10-step-by-step-build-16, eval = TRUE} # Same mechanism as cat_dev, but with other functions diff --git a/11-step-by-step-secure.Rmd b/11-step-by-step-secure.Rmd index 7fd0fc6..d745b0e 100644 --- a/11-step-by-step-secure.Rmd +++ b/11-step-by-step-secure.Rmd @@ -21,7 +21,7 @@ The process of getting your application production-ready implies that the applic With a robust testing suite, you will develop, maintain, and improve in a safe environment and ensure your project sustainability. What will you be testing? Both sides of the application: the business logic and the user interface. -And also, the application load, i.e. how much time and memory are required when your application starts being used by a significant number of users, be it from the user perspective (how long does it take to complete a full scenario) and from the server perspective (how much memory is needed for my app to run). +And also, the application load, i.e. how much time and memory are required when your application starts being used by a significant number of users, be it from the user perspective (how long does it take to complete a full scenario) or from the server perspective (how much memory is needed for my app to run). ### Testing the business logic @@ -37,7 +37,7 @@ To sustain these developments, a lot of tools have been created to secure the de Unit tests are a general concept in software engineering that describes the process of writing a form of assessment to check the validity of your code. A simplified explanation is that if you write a function called `meaning_of_life` that returns `42`, you will expect this function to always return `42`, and to be alerted if ever this value changes. -Using unit tests is a way to secure your work in the future, be it for future you, for your collaborator, or for anybody wanting to collaborate on the project: if anyone comes and change the code behind the `meaning_of_life()` function, and the result is no longer `42`, the developer working on this piece of code will be able to catch it. +Using unit tests is a way to secure your work in the future, be it for future you, for your collaborator, or for anybody wanting to collaborate on the project: if anyone comes and changes the code behind the `meaning_of_life()` function, and the result is no longer `42`, the developer working on this piece of code will be able to catch it. The general idea is to detect bugs and breaking changes at the moment they are happening, not once it is too late. There are several packages in R that can be used to implement unit testing, and you can even implement your own tests. @@ -118,7 +118,7 @@ There are several tools from the web development world that can be used to do ex `puppeteer` is a NodeJS module that drives a Google Chrome headless session and mimics a session on the app. -And good news, there is a Google Chrome extension, called [Puppeteer Recorder](https://chrome.google.com/webstore/detail/puppeteer-recorder/djeegiggegleadkkbgopoonhjimgehda), that allows you to create, while visiting a web page, the `pupepeteer` script to reproduce your visit. +And good news, there is a Google Chrome extension, called [Puppeteer Recorder](https://chrome.google.com/webstore/detail/puppeteer-recorder/djeegiggegleadkkbgopoonhjimgehda), that allows you to create, while visiting a web page, the `puppeteer` script to reproduce your visit. Here is, for example, a very small JavaScript script for testing `{hexmake}` [@R-hexmake], generated by this extension. ``` {.javascript} @@ -147,7 +147,7 @@ For example, typing inside a text input is not recorded: that is completely doab [^step-by-step-secure-2]: See <https://github.com/puppeteer/puppeteer/issues/441> for the code to set the text input values. -Once you have this piece of code, put it into a NodeJS script, and replay the session as many time as you need. +Once you have this piece of code, put it into a NodeJS script, and replay the session as many times as you need. If ever one of the steps cannot be replayed as recorded, the script will fail, notifying you of a regression. Several packages in R mimic what `puppeteer` does (Google Chrome headless orchestration), with notably `{crrri}` [@R-crrri] and `{chromote}` [@R-chromote]. @@ -233,17 +233,17 @@ test$stop() #### B. Monkey test {.unnumbered} -If you are working on a user-facing software (i.e. a software used by external users), there is one rule to live by: every unexpected behavior that can happen, will happen. +If you are working on user-facing software (i.e. software used by external users), there is one rule to live by: every unexpected behavior that can happen, will happen. In other words, if you develop and think "a user will never do that", just expect a user to eventually do "that". But how can we get prepared for the unexpected? -How can we test the "crazy behavior" that user will adopt? -In web development, there exists a methodology called "Monkey testing", which consists of **launching a series of random event on a web page: random text in input, scrolling, clicking, zooming... and see if the application crashes or not**. +How can we test the "crazy behavior" that users will adopt? +In web development, there exists a methodology called "Monkey testing", which consists of **launching a series of random events on a web page: random text in input, scrolling, clicking, zooming... and seeing if the application crashes or not**. This software testing method allows you to test the robustness of the application, by seeing how well it can handle unexpected behaviors. Several JavaScript libraries exist when it comes to monkey testing, one of the most popular (and easy to use) libraries is called [`gremlin.js`](https://github.com/marmelab/gremlins.js). -This library is particularly interesting when it comes to `{shiny}` as it does not need external installation: you can add the library as a bookmark on your browser, navigate to the application, and launch the testing (click on the "Generate Bookmarklet" link on the [top of the README]((https://github.com/marmelab/gremlins.js))). -Figure \@ref(fig:11-step-by-step-secure-7) show an example of running gremlins on the prenoms application. +This library is particularly interesting when it comes to `{shiny}` as it does not need external installation: you can add the library as a bookmark on your browser, navigate to the application, and launch the testing (click on the "Generate Bookmarklet" link on the [top of the README](https://github.com/marmelab/gremlins.js)). +Figure \@ref(fig:11-step-by-step-secure-7) shows an example of running gremlins on the prenoms application. (ref:gremlinscap) Example of using `gremlins.js` on the "prenoms" `{shiny}` application. @@ -251,7 +251,7 @@ Figure \@ref(fig:11-step-by-step-secure-7) show an example of running gremlins o knitr::include_graphics("img/gremlins.png") ``` -And if you want to scale this, you can also combine it with `{shinyloadtest}` [@R-shinyloadtest]: launch a session recording, run `gremlins` one or several time inside the recording, then replay it with multiple sessions. +And if you want to scale this, you can also combine it with `{shinyloadtest}` [@R-shinyloadtest]: launch a session recording, run `gremlins` one or several times inside the recording, then replay it with multiple sessions. With `{crrry}`, this `gremlins` test comes for free: @@ -312,7 +312,7 @@ Then, you can do some changes in your app, and run: shinytest::testApp() ``` -If the `{shinytest}` package detects a visual change in the application, you will be immediately alerted, with a report of the difference from the snapshots you took and the current state of the application. +If the `{shinytest}` package detects a visual change in the application, you will be immediately alerted, with a report of the difference between the snapshots you took and the current state of the application. ### Testing the app load @@ -341,7 +341,7 @@ Sys.sleep(5) # Check that the process is alive p$is_alive() # Open the app in our browser just to be sure -browseURL("http:://localhost:2811") +browseURL("http://localhost:2811") ``` Record the tests, potentially in a new dir: @@ -364,7 +364,7 @@ withr::with_dir( We now have a series of one or more recording/s inside the `shinylogs/` folder: Then, let's switch to our command line, and rerun the session with `shinycannon`. -The `shinycannon` command line tools take several arguments: the path the `.log` file, the URL of the app, `--workers` specify the number of concurrent connections to run, and the `--output-dir` argument specifies where the report should be written. +The `shinycannon` command line tool takes several arguments: the path to the `.log` file, the URL of the app, `--workers` specifies the number of concurrent connections to run, and the `--output-dir` argument specifies where the report should be written. Then, go to your terminal and run: @@ -422,7 +422,7 @@ slt_session_duration(shinyload_runs) <!-- ``` --> -And if you need to bundle everything into an HTML reports, `shinyloadtest_report()` is what you are looking for. +And if you need to bundle everything into an HTML report, `shinyloadtest_report()` is what you are looking for. ```{r 11-step-by-step-secure-22, eval = FALSE} # Generating the report @@ -499,7 +499,7 @@ Sys.sleep(5) # when stopped, and finally the -p flag defines # how to bind the ports of the container # with the ports of the host (left is the host, -# right is the container): in other word, here, +# right is the container): in other words, here, # we bind port 80 of our container to the port 2811 # of our machine. system( @@ -537,9 +537,9 @@ To do that, we can replay our `shinycannon` call, and at the same time use the ` ```{bash 11-step-by-step-secure-29, eval = FALSE} # Replaying the recording shinycannon shinylogs/recording.log \ -# Specificying the host url and the number of "visitors" +# Specifying the host url and the number of "visitors" http://localhost:2811 --workers 10 \ -# Define where the recording will be outputed +# Define where the recording will be outputted --output-dir shinylogs/run3 ``` @@ -547,14 +547,14 @@ Let's launch at the same time a `dockerstats_recurse()`. For example, here, we will print, on each loop, the `MemUsage` of the container, then save the data inside a `dockerstats.csv` file. ```{r 11-step-by-step-secure-30, eval = FALSE} -# Calling recursive the dockerstats function. -# The callback function takes a function, and define +# Calling recursively the dockerstats function. +# The callback function takes a function, and defines # what to do with the data.frame each time the # dockerstats results are computed. dockerstats_recurse( "hexmake", # append_csv is a {dockerstats} function that will - # apped the output to a given csv + # append the output to a given csv callback = append_csv( file = "shinylogs/dockerstats.csv", print = TRUE @@ -570,11 +570,11 @@ Figure \@ref(fig:11-step-by-step-secure-31) shows these processes side to side. knitr::include_graphics("img/hexmake-dockerstats.png") ``` -As you can see, as the number of connections grow, the memory usage grows. +As you can see, as the number of connections grows, the memory usage grows. And we now have a csv with the evolution of the `docker stats` records over time! ```{r 11-step-by-step-secure-32 } -# read_appended_csv() allows to read a csv that has been +# read_appended_csv() allows you to read a csv that has been # constructed with the append_csv() function docker_stats <- read_appended_csv( "shinylogs/dockerstats.csv" @@ -835,7 +835,7 @@ An irrecoverable exception occurred. R is aborting now ... Pretty hard to debug, isn't it? What has actually happened? -On that specific case, it turned out that the package version from `{geojsonsf}` [@R-geojsonsf] was `1.2.1` on our development machine, and the one on the `{shiny}` server was updated to `1.3.0`, and there was a breaking change in the package, as shown in Figure \@ref(fig:11-step-by-step-secure-44). +In that specific case, it turned out that the package version from `{geojsonsf}` [@R-geojsonsf] was `1.2.1` on our development machine, and the one on the `{shiny}` server was updated to `1.3.0`, and there was a breaking change in the package, as shown in Figure \@ref(fig:11-step-by-step-secure-44). This bug was hard to detect as `{geojsonsf}` was not a direct dependency of our app, but a dependency of one of our dependencies, making it slightly more complex to identify. (ref:geojsoncap) Breaking changes in `{geojsonsf}`, a dependency of a dependency of our `{shiny}` application. @@ -844,7 +844,7 @@ This bug was hard to detect as `{geojsonsf}` was not a direct dependency of our knitr::include_graphics("img/geojson.png") ``` -The same thing could have happened if working as a team: one of the computers has an old version, when another one has updated to a more recent one. +The same thing could have happened if working as a team: one of the computers has an old version, while another one has updated to a more recent one. How do we prevent that? This is where the `{renv}` package comes into play: this package allows you to have a project-based library, instead of a global one. In other words, instead of having a library that is global to your machine, `{renv}` allows you to specify packages with fixed versions for a project. @@ -913,7 +913,7 @@ write("library(attempt)", "script.R") Once you want to update your `{renv}` `Lockfile`, call `snapshot()`. ```{r 11-step-by-step-secure-50, eval = FALSE} -# Snapshoting the current status of the environment +# Snapshotting the current status of the environment renv::snapshot(confirm = FALSE) ``` @@ -947,7 +947,7 @@ If you want to know more, we invite you to refer to the [official documentation] #### A. R, Docker, `{shiny}` {.unnumbered} -Docker is a program that allows to download, install, create, launch and stop multiple operating systems, called containers, on a machine, which will be called the host. +Docker is a program that allows you to download, install, create, launch and stop multiple operating systems, called containers, on a machine, which will be called the host. This host can be your local computer, or the server where you deploy your application/s. Docker was designed for **enclosing software environments inside an image that can later be launched**. @@ -1086,7 +1086,7 @@ Developers have their own R versions and operating systems, which generally diff If you plan on using Docker as a deployment mechanism, you can also use Docker as a local developer environment. Thanks to the containers maintained by the [The Rocker Project](https://www.rocker-project.org/), it's possible to have a local environment that comes close to what you will find on the production server. -What's even more interesting is that this project offers images that can contain RStudio server: that means that the application that you will deploy in production can have the very same configuration as the one developers are using on their local machine: thanks to these containers, developers can work on a version of R that matches the one from the production server, using packages that will exactly match the one used in production. +What's even more interesting is that this project offers images that can contain RStudio server: that means that the application that you will deploy in production can have the very same configuration as the one developers are using on their local machine: thanks to these containers, developers can work on a version of R that matches the one from the production server, using packages that will exactly match the ones used in production. Even more interesting is using RStudio inside Docker in combination with `{renv}`: the developers work on their machines, inside an IDE they know, and with system requirements (R versions, packages, etc.) that can be reproduced on the production server! diff --git a/12-secure.Rmd b/12-secure.Rmd index 3106f53..e8f7c99 100644 --- a/12-secure.Rmd +++ b/12-secure.Rmd @@ -109,7 +109,7 @@ Even when working alone. If you are working with a remote tool with a graphical interface like GitLab, GitHub or Bitbucket, there is a good chance you will be using issues. Issues are "notes" or "tickets" that can be used to track a bug or to suggest a feature. This tool is crucial when it comes to project management: issues are the perfect spot for organizing and discussing ideas, but also to have an overview of what has been done, what is currently being done, and what is left to be done. -Issue may also be used as a discussion medium with beta testers, clients or sponsors. +Issues may also be used as a discussion medium with beta testers, clients or sponsors. One other valuable feature of issues is that they can be referenced inside commits using a hashtag and its number: `#123`. In other words, when you send code to the centralized server, you can link this code to one or more issues and corresponding commits appear in the issue discussions. @@ -122,7 +122,7 @@ In other words, when you send code to the centralized server, you can link this If you are using RStudio, you will find a pull/push button, a stage and commit interface, and a tool for visualizing differences in files. Everything you need to get started is there. -Note that of course, it will be better in the long run to get a more complete understanding of how `Git` works, so that when things get more complexe, you will be able to handle them. +Note that of course, it will be better in the long run to get a more complete understanding of how `Git` works, so that when things get more complex, you will be able to handle them. ### As part of a larger world @@ -199,12 +199,12 @@ If you want to learn more about `Git`, here are some resources that have helped We have seen in Chapter \@ref(build-yourself-safety-net) how to build a testing infrastructure for your app, notably using the `{testthat}` [@R-testthat] package. What we have described is a way to build it locally, before running your test on your own machine. But there is a big flaw to this approach: you have to remember to run the tests, be it regularly or before making a pull request/pushing to the server. -To do this kind of job, you will be looking for **a tool to do automated testing at the repository level: in other words, a software that can test your application whenever a piece of code is pushed/moved on the repository**. +To do this kind of job, you will be looking for **a tool to do automated testing at the repository level: in other words, software that can test your application whenever a piece of code is pushed/moved on the repository**. To do this, various tools are available, each with their own features. Here is a non-exhaustive list of the ones you can choose: -[Travis CI](https://travis-ci.org/) is a software that can be synced with your `Git` repositories (GitHub or Bitbucket), and whenever something happens on the repo, the events described in the travis configuration file (`.travis.yml`) are executed. +[Travis CI](https://travis-ci.org/) is software that can be synced with your `Git` repositories (GitHub or Bitbucket), and whenever something happens on the repo, the events described in the travis configuration file (`.travis.yml`) are executed. If they exit with a code 0, the test passes. If they do not, the integrated tests have failed. Travis CI integration may be used internally and externally: internally, in the sense that before merging any pull request, the project manager has access to a series of tests that are automatically launched. diff --git a/13-deploy.Rmd b/13-deploy.Rmd index 5049d04..4086ed0 100644 --- a/13-deploy.Rmd +++ b/13-deploy.Rmd @@ -21,7 +21,7 @@ Here is a quick checklist of things to think about once your application is read - [ ] Everything is fully documented. -- [ ] Test coverage is good, i.e. you cover a sufficient amount of the codebase, and these tests cover the core/strategic algorithms +- [ ] Test coverage is good, i.e. you cover a sufficient amount of the codebase, and these tests cover the core/strategic algorithms. - [ ] Everyone in the project knows the person to call if something goes wrong. @@ -60,19 +60,19 @@ Then, try the `run_app()` function to check that the app can be launched. #### A. Local build {.unnumbered} -Building an app as a package also means that this app can be bundled into an archive, and then shared, either as is or using a package repository like the CRAN. +Building an app as a package also means that this app can be bundled into an archive, and then shared, either as is or using a package repository like CRAN. -To do that, you first need an bundled version of your app, which can be created using the `build()` function from `{pkgbuild}` [@R-pkgbuild] in the same working directory as your application. +To do that, you first need a bundled version of your app, which can be created using the `build()` function from `{pkgbuild}` [@R-pkgbuild] in the same working directory as your application. Calling this function will create a .tar.gz file that is called `mygolem_0.0.1.tar.gz` (of course with the name of your package). Once you have this `tar.gz`, you can send it to your favorite package repository. You can also share the file as is with others. -If you do so, they will have to install the app with `remotes::install_local("path/to/tar.gz")`, that will take care of doing a full installation of the app, including installing the required dependencies. +If you do so, they will have to install the app with `remotes::install_local("path/to/tar.gz")`, which will take care of doing a full installation of the app, including installing the required dependencies. Then, they can do `library(yourpackagename)` and `run_app()` on their machine. #### B. Send to a package repository {.unnumbered} -The upside of building the application `{golem}`, i.e. as a package, is that you can share your application on a remote package manager, the more widely used, for example, on the CRAN like `{dccvalidator}` [@R-dccvalidator], or on BioConductor like `{spatialLIBD}` [@R-spatialLIBD]. +The upside of building the application with `{golem}`, i.e. as a package, is that you can share your application on a remote package manager, the most widely used, for example, on CRAN like `{dccvalidator}` [@R-dccvalidator], or on BioConductor like `{spatialLIBD}` [@R-spatialLIBD]. But any other package manager will work: for example, if the company uses RStudio Package Manager, your application can be installed here in the same way as any other package. If your application is open source, the package structure also allows you to install from GitHub, by using the `remotes::install_github()` function.[^deploy-with-golem-1] For example, this is what you can do with `{hexmake}` or `{tidytuesday}`: as they are open-source packages, they can be installed from GitHub. @@ -110,11 +110,11 @@ At the time of writing this book, there are two main ways to deploy a shiny app ### RStudio environments -RStudio proposes three services to deploy `{shiny}` application: +RStudio proposes three services to deploy `{shiny}` applications: -- `shinyapps.io`, an on-premises solution, can serve `{shiny}` application (freemium). +- `shinyapps.io`, an on-premises solution, can serve `{shiny}` applications (freemium). -- `Shiny Server` is a software you have to install on your own server, and can be used to deploy multiple applications (you can find either an open source or a professional edition). +- `Shiny Server` is software you have to install on your own server, and can be used to deploy multiple applications (you can find either an open source or a professional edition). - `RStudio Connect` is a server-based solution that can deploy `{shiny}` applications and `Markdown` documents (and other kinds of content), and serves them as ordinary websites. @@ -126,16 +126,16 @@ Each of these platforms has its own function to create an `app.R` file that is t - `golem::add_shinyserver_file()` -These `app.R` files call a `pkgload::load_all()` function, that will mimic the launch of your package, and then call the `run_app()` function from your packaged app. +These `app.R` files call a `pkgload::load_all()` function, which will mimic the launch of your package, and then call the `run_app()` function from your packaged app. Note that if you need to configure the way your app is launched on these platforms (for example, if you need to pass arguments to the `run_app()` function), you will have to edit this file. Note that when using these functions, you will be able to use the "One click deploy" for these platforms: on the top right of these `app.R`, use the Blue Button to deploy to a server. -Another way to deploy your `{golem}`-based app to `{shiny}` server and to Connect is to link these two software to a local repository (for example, an RStudio Package Manager), and then to only use `mypackage::run_app()` to the `app.R`. +Another way to deploy your `{golem}`-based app to `{shiny}` server and to Connect is to link these two pieces of software to a local repository (for example, an RStudio Package Manager), and then to only use `mypackage::run_app()` in the `app.R`. ### Docker -Docker is an open source software used to build and deploy applications in containers. +Docker is open source software used to build and deploy applications in containers. Docker has become a core solution in the DevOps world and a lot of server solutions are based on it. See Part 5, "Strengthen", for a more complete introduction to Docker. diff --git a/14-when_optimize.Rmd b/14-when_optimize.Rmd index e36b7a9..885d173 100644 --- a/14-when_optimize.Rmd +++ b/14-when_optimize.Rmd @@ -20,7 +20,7 @@ That **focusing on optimizing small portions of your app before making it work f Why? Here is the general idea: let's say the schema in Figure \@ref(fig:14-when-optimize-1) represents your software, and its goal is to make things travel from *X1* to *X2*, but you have a bottleneck at *U*. You are building elements piece by piece: first, the portion `X1.1` of the "road", then `X1.2`, etc. -Only when you have your application ready can you really appreciate where your bottleneck is, and you can focus on making things go fast from `X1.1` to `X.1.2`, these performance gains won't make your application go faster: you will only make the elements move faster to the bottleneck. +Only when you have your application ready can you really appreciate where your bottleneck is, and you can focus on making things go fast from `X1.1` to `X1.2`, these performance gains won't make your application go faster: you will only make the elements move faster to the bottleneck. When? Once the application is ready: here in our example, we can only detect the bottleneck once the full road is actually built, not while we are building the circle. @@ -75,7 +75,7 @@ Yet you will only realize that once the application is up and running! ### Don't sacrifice readability As said in the last section, every piece of code can be rewritten to be faster, either from R to R or using a lower-level language: for example C or C++. -You can also rebuild data manipulation code switching from one package to another, or use a complex data structures to optimize memory usage, etc. +You can also rebuild data manipulation code switching from one package to another, or use complex data structures to optimize memory usage, etc. But that comes with a price: **not keeping things simple for the sake of local optimization makes maintenance harder, even more if you are using a lesser-known language/package**. Refactoring a piece of code is better done when you keep in mind that "the primary goal should be to produce human-friendly code, even at the cost of your original design. If the laser focus is on the solution rather than the process, there's a greater chance your application will end up more contrived and complicated than it was in the first place" [@lemaire2020]. @@ -140,7 +140,7 @@ The best way to profile R code is by using the `{profvis}` [@R-profvis] package, With `{profvis}`, you can spot the bottleneck in your function. Without an automated tool to do the profiling, the developers would have to profile by guessing, which will, most of the time, come with bad results: -[^when_optimize-1]: `{utils}` also comes with a function call `Rprof()`, but we will not be examining this one here, as `{profvis}` provides a more user-friendly and enhanced interface to this profiling function. +[^when_optimize-1]: `{utils}` also comes with a function called `Rprof()`, but we will not be examining this one here, as `{profvis}` provides a more user-friendly and enhanced interface to this profiling function. > One of the lessons that the original Unix programmers learned early is that intuition is a poor guide to where the bottlenecks are, even for one who knows the code in question intimately. > @@ -193,7 +193,7 @@ profvis({ What you see now is called a `flame graph`: it is a detailed timing of how your function has run, with a clear decomposition of the call stack. What you see in the top window is the expression evaluated, and on the bottom the details of the call stack, with what looks a little bit like a Gantt diagram. -This result reads as follow: the wider the function call, the more time it has taken R to compute this piece of code. +This result reads as follows: the wider the function call, the more time it has taken R to compute this piece of code. On the very bottom, the "top" function (i.e. the function which is directly called in the console), and the higher you go, the more you enter the nested function calls. Here is how to read the graph in \@ref(fig:14-when-optimize-4): @@ -224,7 +224,7 @@ If you are working on profiling the memory usage, you can also use the `{profmem ```{r 14-when-optimize-6 } library(profmem) -# Computing the memory used by each c +# Computing the memory used by each call p <- profmem({ x <- raw(1000) A <- matrix(rnorm(100), ncol = 10) @@ -262,13 +262,13 @@ And of course, without a clear documentation of what we are doing, we will be mi In other words, if you want to be sure that you are actually optimizing, be sure that you have a basis for comparison. How to do that? -One thing that can be done is to keep an RMarkdown file with your starting point: use this notebook to keep track of what you are doing, by noting where you are starting from (i.e, what's the original function you want to optimize), and compare it with the new one. -By using an Rmd, you can document the strategies you have been using to optimize the code, e.ga: "switched from for loop to vectorize function", "changed from x to y", etc. +One thing that can be done is to keep an RMarkdown file with your starting point: use this notebook to keep track of what you are doing, by noting where you are starting from (i.e., what's the original function you want to optimize), and compare it with the new one. +By using an Rmd, you can document the strategies you have been using to optimize the code, e.g.: "switched from for loop to vectorized function", "changed from x to y", etc. This will also be helpful for the future: either for you in other projects (you can get back to this document), or for other developers, as it will explain why specific decisions have been made. To do the timing computation, you can use the `{bench}` [@R-bench] package, which compares the execution time (and other metrics) of two functions. This function takes a series of named elements, each containing an R expression that will be timed. -Note that by default, the `mark()` function compares the output of each function, +Note that by default, the `mark()` function compares the output of each function. Once the timing is done, you will get a data.frame with various metrics about the benchmark. @@ -294,7 +294,7 @@ res <- bench::mark( res ``` -Here, we have an empirical evidence that one code is faster than the other: by benchmarking the speed of our code, we are able to determine which function is the fastest. +Here, we have empirical evidence that one code is faster than the other: by benchmarking the speed of our code, we are able to determine which function is the fastest. If you want a graphical analysis, `{bench}` comes with an `autoplot` method for `{ggplot2}` [@R-ggplot2], as shown in Figure \@ref(fig:14-when-optimize-10): @@ -333,7 +333,7 @@ Yet many, if not all, studies of how the web is browsed report the same results: [^when_optimize-3]: broadbandsearch <https://www.broadbandsearch.net/blog/mobile-desktop-internet-usage-statistics> for example, reports a 53.3% share for mobile browsing. -And, the advantages of running it in your browser is that it can perform the analysis on locally deployed applications: in other words, you can launch your `{shiny}` application in your R console, open the app in Google Chrome, and run the audit. +And, the advantage of running it in your browser is that it can perform the analysis on locally deployed applications: in other words, you can launch your `{shiny}` application in your R console, open the app in Google Chrome, and run the audit. A lot of online services need a URL to do the audit! Each result from the audit comes with advice and changes you can make to your application to make it better, with links to know more about the specific issue. @@ -343,7 +343,7 @@ It is always a good mood booster to see our app passing some audited points! Here is a quick introduction to this tool: -- Open Chrome in incognito mode (File \> New Icognito Window),[^when_optimize-4] so that the page performance is not influenced by any of the installed extensions in your Google Chrome. +- Open Chrome in incognito mode (File \> New Incognito Window),[^when_optimize-4] so that the page performance is not influenced by any of the installed extensions in your Google Chrome. - Open your developer console, either by going to View \> Developer \> Developer tools, by right-clicking \> Inspect, or with the keyboard shortcut ctrl/cmd + alt + I, as shown in Figure \@ref(fig:14-when-optimize-11). - Go to the "Audit" tab. - Configure your report (or leave the default). @@ -384,11 +384,11 @@ Once the audit is finished, you have some basic but useful indications about you - SEO, search engine optimization, or how your app will perform when it comes to search engine indexation.[^when_optimize-6] - Progressive Web App (PWA): A PWA is an app that can run on any device, *"reaching anyone, anywhere, on any device with a single codebase"*. - Google audit your application to see if your application fits with this idea. + Google audits your application to see if your application fits with this idea. [^when_optimize-6]: Search engine indexation refers to how Google ranks your website in the search results for a given query. -Profiling web page is a wide topic and a lot of things can be done to enhance the global page performance. +Profiling web pages is a wide topic and a lot of things can be done to enhance the global page performance. That being said, if you have a limited time to invest in optimizing the front-end performance of the application, Google Lighthouse is a perfect tool, and can be your go-to audit tool for your application. And if you want to do it from R, the npm lighthouse module allows you to output the audit in JSON, which can then be brought back to R! @@ -399,7 +399,7 @@ lighthouse --output json \ http://localhost:2811 ``` -Then, being a JSON file, you can call if from R: +Then, being a JSON file, you can call it from R: ```{r 14-when-optimize-13 } # Reading the JSON output of your lighthouse audit, @@ -408,7 +408,7 @@ lighthouse_report <- jsonlite::read_json("data-raw/output.json") lighthouse_report$audits$`speed-index`$displayValue ``` -The results are contained in the `audits` sections of this object, and each of these sub-elements contains a `description` field, detailing what the metric means. +The results are contained in the `audits` section of this object, and each of these sub-elements contains a `description` field, detailing what the metric means. Here are, for example, some of the results, focused on performance, with their respective descriptions: @@ -550,7 +550,7 @@ To minify JavaScript, HTML and CSS files from R, you can use the `{minifyr}` [@R For example, compare the size of this file from `{shiny}`: ```{r 14-when-optimize-24, eval = FALSE} -# Displaying the file size of a CSS file from {shiny} +# Displaying the file size of a JS file from {shiny} fs::file_size( system.file("www/shared/shiny.js", package = "shiny") ) @@ -563,7 +563,7 @@ cat("239K") To its minified version: ```{r 14-when-optimize-26, eval = FALSE} -# Using the {minifyr} package to minify the CSS file +# Using the {minifyr} package to minify the JS file minified <- minifyr::minifyr_js_gcc( system.file("www/shared/shiny.js", package = "shiny"), "shinymini.js" @@ -585,12 +585,12 @@ Of course, minification will not suddenly make your application blazing fast, bu Minification can be important notably if you expect your audience to be connecting to your app with a low bandwidth: whenever your application starts, the browser has to download the source files from the server, meaning that the larger these files, the longer it will take to render. Note that `{shiny}` files are minified by default, so you will not have to re-minify them. -But most packages that extend `{shiny}` are not, so minifying the CSS and JavaScript files from these packages might help you win some points on you Google Lighthouse report! +But most packages that extend `{shiny}` are not, so minifying the CSS and JavaScript files from these packages might help you win some points on your Google Lighthouse report! To do this automatically, you can add the `{minifyr}` commands to your deployment, be it on your CD/CI platform, or as a Dockerfile step. `{minifyr}` comes with a series of functions to do that: -- `minify_folder_css()`, `minify_folder_js()`, `minify_folder_html()` and `minify_folder_json()` do a bulk minification of the files found in a folder that matches the extension. +- `minify_folder_css()`, `minify_folder_js()`, `minify_folder_html()` and `minify_folder_json()` do a bulk minification of the files found in a folder that match the extension. - `minify_package_js()`, `minify_package_css()`, `minify_package_html()` and `minify_package_json()` will minify the CSS and JavaScript files contained inside a package installed on the machine. Here is what it can look like inside a `Dockerfile` (Note that you will need to install NodeJS inside the container): diff --git a/15-common-app-caveats.Rmd b/15-common-app-caveats.Rmd index d9a16b2..7074eb9 100644 --- a/15-common-app-caveats.Rmd +++ b/15-common-app-caveats.Rmd @@ -18,13 +18,13 @@ library(shiny) library(lubridate) ui <- function(){ tagList( - # Adding a first input which allow + # Adding a first input which allows us # to select a specific date dateInput( "date", "choose a date" ), - # Adding a second input allowing + # Adding a second input allowing us # to specify a year selectInput( "year", @@ -39,7 +39,7 @@ server <- function( output, session ){ - # We want the year to be update whenever + # We want the year to be updated whenever # the dateInput is updated observeEvent( input$date , { updateSelectInput( @@ -49,7 +49,7 @@ server <- function( ) }) - # We want the date to be update whenever + # We want the date to be updated whenever # the selectInput is updated observeEvent( input$year , { updateDateInput( @@ -108,7 +108,7 @@ server <- function(input, output, session) { }) observeEvent(input$year, { - # Preventing this update to be sent at application launch + # Preventing this update from being sent at application launch if (input$year != format(input$date, "%Y")) { date <- as.Date(ISOdate(input$year, 1, 1)) message("Changing date to ", date) @@ -123,14 +123,14 @@ shinyApp(ui, server) ### `observe` vs `observeEvent` One of the most common features of reactive inferno is the use of `observe()` in cases where you should use `observeEvent`. -Spoiler: you should try to use `observeEvent()` as much as possible, and avoid `observe()`as much as possible. +Spoiler: you should try to use `observeEvent()` as much as possible, and avoid `observe()` as much as possible. At first, `observe()` seems easier to implement, and feels like a shortcut as you do not have to think about what to react to: everything gets updated without you thinking about it. But the truth is, this stairway does not lead to heaven. Let's stop and think about `observe()` for a minute. This function updates **every time a reactive object it contains is invalidated**. -Yes, this works well if you have a small number of reactive objects in the observer, but that gets tricky when you start adding a long list of things inside your `observe()`, as you might be launching a computation 10 times if your reactive scope contains 10 reactive objects that are somehow invalidated in chain. +Yes, this works well if you have a small number of reactive objects in the observer, but that gets tricky when you start adding a long list of things inside your `observe()`, as you might be launching a computation 10 times if your reactive scope contains 10 reactive objects that are somehow invalidated in a chain. And believe us, we have seen pieces of code where the `observe()` contains hundreds of lines of code, with reactive objects all over the place, with one `observe()` context being invalidated dozens of times when one input changes in the application. For example, let's start with that: @@ -192,7 +192,7 @@ server <- function(input, output, session){ i <<- i + 1 # We print the i value to the console cat_rule(as.character(i)) - # If the user select lower, then the text is + # If the user selects lower, then the text is # passed through tolower, otherwise it's passed # through toupper if (input$casefolding == "lower") { @@ -239,10 +239,10 @@ server <- function(input, output, session){ # Use input_txt as a container for our input input_txt <- input$txt if (input$rev){ - # If the input$rev is select, we reverse the text + # If the input$rev is selected, we reverse the text input_txt <- stri_reverse(input_txt) } - # If the user select lower, then the text is + # If the user selects lower, then the text is # passed through tolower, otherwise it's passed # through toupper if (input$casefolding == "lower") { @@ -290,13 +290,13 @@ server <- function(input, output, session){ # We print the i value to the console cat_rule(as.character(i)) if (input$rev){ - # If the input$rev is select, we reverse the text + # If the input$rev is selected, we reverse the text r$input_txt <- stri_reverse(r$input_txt) } else { # Otherwise, we leave it as it is r$input_txt <- input$txt } - # If the user select lower, then the text is + # If the user selects lower, then the text is # passed through tolower, otherwise it's passed # through toupper if (input$casefolding == "lower") { @@ -328,7 +328,7 @@ That's why (well, in 99% of cases), it is safer to go with `observeEvent`, as it Then, if a reactive context is invalidated, **you know why**. For example, here is where the reactive invalidation can happen (lines with a `*`)[^common-app-caveats-1]: -[^common-app-caveats-1]: Of course it's an over-simplification: the reactive context will not be invalidated in all of these contexts. The idea is to illustrate how `observe()` can lead to invalidation points that are spread all across the code bloc. +[^common-app-caveats-1]: Of course it's an over-simplification: the reactive context will not be invalidated in all of these contexts. The idea is to illustrate how `observe()` can lead to invalidation points that are spread all across the code block. ``` {.r} observe({ @@ -399,7 +399,7 @@ The idea here is to allow complete control over when the image is recomputed: on That might seem like a lot of extra work, but that is definitely worth considering in the long run, as it will help in optimizing the rendering (fewer computations), and lowering the number of errors that can result from too much reactivity inside an application. Here is a small example of this implementation, using an environment to store the value. -When using this pattern, we do not rely on any reactive value invalidating the reactive context: the second result is only displayed when the `"render2"` flag is triggered, giving us a full control on how the reactivity is propagated. +When using this pattern, we do not rely on any reactive value invalidating the reactive context: the second result is only displayed when the `"render2"` flag is triggered, giving us full control over how the reactivity is propagated. ```{r 15-common-app-caveats-8, eval = FALSE} library(shiny) @@ -547,7 +547,7 @@ server <- function( output, session ){ - # We start by creating a new instance of th + # We start by creating a new instance of the class r6 <- MyDataProcessing$new() # Passing this object to the two server functions mod_data_cleaning_server("data_cleaning_ui_1", r6) @@ -606,7 +606,7 @@ test_that("R6 Class works", { }) ``` -Using R6 allows to rely on these battle-tested tools when it comes to testing functions, something which is made more complex when using other patterns like `reactiveValues()`. +Using R6 allows us to rely on these battle-tested tools when it comes to testing functions, something which is made more complex when using other patterns like `reactiveValues()`. ### Logging reactivity with `{whereami}` @@ -621,7 +621,7 @@ whereami::cat_where( whereami::whereami() ) ── Running server(...) at app_server.R#9 (2) ─────────────── -Combining `cat_where()` will implement a reactive logging to your console while developing: that way, you can instantaneously know what reactive contexts are invalidated while using the application. +Combining `cat_where()` will implement reactive logging to your console while developing: that way, you can instantaneously know what reactive contexts are invalidated while using the application. Of course, you still have to implement it by hand, but that is definitely worth the effort: seeing in real time, in your console, which line is run allows you to detect unexpected behavior. For example, you will be able to see that the `observeEvent()` from `mod_main.R#79` has been called 17 times when launching the app, which might be an unexpected behavior. @@ -650,9 +650,9 @@ knitr::include_graphics("img/plot_whereami.png") There are many reasons we would want to change things on the UI based on what happens in the server: changing the choices of a `selectInput()` based on the columns of a table which is uploaded by the user, showing and hiding pieces of the app according to an environment variable, allowing the user to create an indeterminate number of inputs, etc. Chances are that to do that, you have been using the `uiOutput()` and `renderUI()` functions from `{shiny}` [@R-shiny]. -Even if convenient, and the functions of choice in some specific context, this pair of functions makes R do a little bit too much: you are making R regenerate the whole UI component instead of changing only what you need, which can be a suboptimal, be it from the user point of view, or from a developer perspective. +Even if convenient, and the functions of choice in some specific context, this pair of functions makes R do a little bit too much: you are making R regenerate the whole UI component instead of changing only what you need, which can be suboptimal, be it from the user point of view, or from a developer perspective. -**One of the instance in which this pattern might not be optimal is in the case where your visitors do not have an high-speed internet or when visiting and using a smartphone, contexts where every byte counts**. +**One of the instances in which this pattern might not be optimal is in the case where your visitors do not have a high-speed internet or when visiting and using a smartphone, contexts where every byte counts**. Rendering large elements from the server side in your `{shiny}` app means that these elements will have to transit through the socket, i.e. they need to be sent by the server, and downloaded by the browser. In this case, the smaller the message size the better! @@ -678,7 +678,7 @@ ui <- function(){ actionButton( "change", "show/hide graph", - # The toggle() function hide or show the queried element + # The toggle() function hides or shows the queried element onclick = "$('#plot').toggle()" ), plotOutput("plot") @@ -805,7 +805,7 @@ That is a full second for rendering four of them, while R should be busy doing s #### B. `update*` inputs {.unnumbered} -Almost every `{shiny}` input, even the custom ones from packages, come with an `update_` function that allows us to change the input values from the server side, instead of re-creating the UI entirely. +Almost every `{shiny}` input, even the custom ones from packages, comes with an `update_` function that allows us to change the input values from the server side, instead of re-creating the UI entirely. For example, here is a way to update the content of a `selectInput` from the server side: ```{r 15-common-app-caveats-19, eval = FALSE} @@ -814,7 +814,7 @@ ui <- function(){ tagList( # We start the selectInput empty selectInput("species", "Species", choices = NULL), - # The selectInput will be populate + # The selectInput will be populated # when the update button is pressed actionButton("update", "Update") ) @@ -903,8 +903,8 @@ Because these systems have been created to handle and manipulate data on disk: i For example, if you have a `selectInput()` that is used to perform a filter on a dataset, you can do that filter straight inside SQL, instead of bringing all the data to R and then doing the filter. That is even more necessary if you are building the app for a large number of users: for example if one `{shiny}` session takes up to 300MB, multiply that by the number of users that will need one session, and you will have a rough estimate of how much RAM you will need. -On the contrary, if you reduce the data manipulation so that it is done by the back-end, you will have, let's say, one database with 300MB of data, so the database size will remain (more or less constant), and the only RAM used by `{shiny}` will be the data manipulation, not the data storage. -That's even more true now that almost any operation you can do today in `{dplyr}` [@R-dplyr] would be doable with an SQL back-end, and that is the purpose of the `{dbplyr}` [@R-dbplyr] package: translates `{dplyr}` code into SQL. +On the contrary, if you reduce the data manipulation so that it is done by the back-end, you will have, let's say, one database with 300MB of data, so the database size will remain (more or less) constant, and the only RAM used by `{shiny}` will be the data manipulation, not the data storage. +That's even more true now that almost any operation you can do today in `{dplyr}` [@R-dplyr] would be doable with an SQL back-end, and that is the purpose of the `{dbplyr}` [@R-dbplyr] package: translating `{dplyr}` code into SQL. If using a database as a back-end seems a little bit far-fetched right now, that is how it is done in most programming languages: if you are building a web app with NodeJS or Python for example, and need to interact with data, nothing will be stored in RAM: you will be relying on an external database to store your data. Then your application will be used to make queries to this database back-end. @@ -963,17 +963,17 @@ Databases come with APIs and drivers that help retrieve and transfer data: be it Using a database is one of the solutions for making your app smaller and more efficient in the long run, especially if you need to scale your app to thousands of visitors. Indeed, **if you plan on having your app scale to numerous people, that will mean that a lot of R processes will be triggered. And if your data is contained in your app, this will mean that each R process will take a significant amount of RAM if the dataset is large**. For example, if your dataset alone takes \~300 MB of RAM, that means that if you want to launch the app 10 times, you will need \~3GB of RAM. -On the other hand, if you decide to switch these data to an external database, it will lower the global RAM need: the DB will take these 300MB of data, and each shiny application will make a request to the database. +On the other hand, if you decide to switch these data to an external database, it will lower the global RAM need: the DB will take these 300MB of data, and each shiny application will make a request to the database. For instance, if the database needs 300MB, and one shiny app 50MB, then 10 apps will be 300MB (for the DB) + 50MB \* 10 (for the 10 apps). In practice, other things are to be considered: making database requests can be computationally expensive, and might need some network adjustments, but you get the idea. -How does one choose between database back-end? -Well, first of all you need to see what is available in the environment the application will be deployed: maybe the company you are building the application for already has database servers deployed. +How does one choose between database back-ends? +Well, first of all you need to see what is available in the environment the application will be deployed in: maybe the company you are building the application for already has database servers deployed. If ever you are free to choose any database as a back-end, your choice should be driven by what kind of operations you want to make on these databases. **For example, SQL databases are designed to store tabular data, and they tend to be very fast when it comes to reading data: so if you have one or more large data.frames you want to use inside your application, and with no specific update of these data, an SQL back-end can be the perfect choice**. On the other hand, a NoSQL database like MongoDB will be faster when it comes to doing write operations, and can store any kind of object: for example, `{hexmake}` can use a MongoDB back-end to store RDS files. But that comes with a price: read calls are a little bit slower, and you might have to work a little bit more on handling the JSON results that come out of MongoDB. -Another example of an app that uses on an external database is `{databasedemo}`, available at [engineering-shiny.org/databasedemo/](https://engineering-shiny.org/databasedemo/). +Another example of an app that uses an external database is `{databasedemo}`, available at [engineering-shiny.org/databasedemo/](https://engineering-shiny.org/databasedemo/). Feel free to follow this link for more information about this application! Covering all the available types of databases and the packages associated with each is a very, very large topic: there are dozens of database systems, and as many (if not more) packages to interact with them. @@ -983,7 +983,7 @@ For more extensive coverage of using databases in R, please follow these resourc - [colinfay/r-db](https://colinfay.me/r-db/), a Docker image that bundles the toolchain for a lot of database systems for R. -- [CRAN Task View: Databases with R](https://cran.r-project.org/web/views/Databases.html): the official task view from CRAN with a series of packages for database manipulation +- [CRAN Task View: Databases with R](https://cran.r-project.org/web/views/Databases.html): the official task view from CRAN with a series of packages for database manipulation. ### Data-source checklist diff --git a/16-optimizing-shiny-code.Rmd b/16-optimizing-shiny-code.Rmd index bc477dd..21b5f99 100644 --- a/16-optimizing-shiny-code.Rmd +++ b/16-optimizing-shiny-code.Rmd @@ -2,14 +2,14 @@ ## Optimizing R code -In its core, `{shiny}` runs R code on the server side. +At its core, `{shiny}` runs R code on the server side. To be efficient, the R code computing your values and returning results also has to be optimized. Optimizing R code is a very broad topic, and it would be possible to write a full book about it. In fact, a lot of books and blog posts already cover this topic. Instead of re-writing these books, we will try to point to some crucial resources you can refer to if you want to get started optimizing your R code. -- Efficient R programming [@colingillespie2017], has a series of methods you can quickly put into practice for more efficient R code. +- Efficient R programming [@colingillespie2017] has a series of methods you can quickly put into practice for more efficient R code. - Advanced R [@hadleywickham2019] has a chapter about optimizing R code (number 24). In the rest of this chapter, we will be focusing on how to optimize `{shiny}` specifically. @@ -18,7 +18,7 @@ Instead of re-writing these books, we will try to point to some crucial resource ### What is caching? -Caching is the process of storing resources-intensive results so that when they are needed again, your program can reuse the result another time without having to redo the computation again. +Caching is the process of storing resource-intensive results so that when they are needed again, your program can reuse the result another time without having to redo the computation again. This is particularly useful for computation that will always return the same result, and should never be used if you expect the result could vary from one function call to the other. How does it work? @@ -38,10 +38,10 @@ The downside is that you only have limited space on your screen: when your scree In the context of an interactive application built with `{shiny}`, it makes sense to cache data structures: users tend to repeat what they do, or go back and forth between parameters. For example, if you have a graph which is taking 2 seconds to render (which is quite common in `{shiny}`, notably when relying on `{ggplot2}` [@R-ggplot2]), you do not want these 2 seconds to be repeated over and over again when users switch from one parameter to another. -In that case, it does make sense to cache the result: if you call `ploting_function(input$selection)` twice with the same value for `input$selection`, and you are sure that this plot will be the same every time, you can cache it. +In that case, it does make sense to cache the result: if you call `plotting_function(input$selection)` twice with the same value for `input$selection`, and you are sure that this plot will be the same every time, you can cache it. In other words, instead of recomputing the graph on each `input$selection` change, you can cache the plot the first time it is generated, and then the application will read the cache instead of re-doing the computation. -Same goes for queries to a database: if a query is done with the same parameters, and you know that they will return the same result, there is no need to ask the database again and again—ask the cache to retrieve the data. +Same goes for queries to a database: if a query is done with the same parameters, and you know that it will return the same result, there is no need to ask the database again and again—ask the cache to retrieve the data. Keep in mind that this caching mechanism is only to be used when **the data don't change**. For example, if you are calling a database which is updated on a regular basis, you might not want to cache the results of a function. @@ -50,7 +50,7 @@ In that specific case, you will want the query to be performed every time the fu ### Native caching in R At least two packages in R implement caching of functions (also called memoization): `{R.cache}` [@R-R.cache], and `{memoise}` [@R-memoise]. -They both more or less work the same way: you will call a memoization function on another function, and cache is created for this function output, based on the arguments value. +They both more or less work the same way: you will call a memoization function on another function, and a cache is created for this function's output, based on the arguments' values. Then every time you call this function again with the same parameters, the cache is returned instead of computing the function another time. For example, if computing your data once takes 5 seconds with the parameter `n = 50`, the next time you will be calling this function with `n = 50`, instead of recomputing, R will go and fetch the value stored in cache. @@ -60,7 +60,7 @@ Here is a simple example with `{memoise}`: library(memoise) library(tictoc) # We define a function that sleeps for a given number of seconds, -# then return the time +# then returns the time sleep_and_return_time <- function(seconds = 1){ Sys.sleep(seconds) return(Sys.time()) @@ -69,7 +69,7 @@ sleep_and_return_time <- function(seconds = 1){ msleep_and_return_time <- memoise(sleep_and_return_time) # We use the {tictoc} package to count the time to run the code tic() -# This will sleeep for 2 seconds and return the time +# This will sleep for 2 seconds and return the time msleep_and_return_time(2) # The code should have taken around 2 seconds to run toc() @@ -285,7 +285,7 @@ Then, it can be used in an app: library(shiny) ui <- function(){ tagList( - # The user can select one of the cut from ggplot2::diamonds, + # The user can select one of the cuts from ggplot2::diamonds, # {shiny} will then query the SQL database to retrieve the # first rows of the result selectInput("cut", "cut", unique(ggplot2::diamonds$cut)), @@ -301,7 +301,7 @@ server <- function( # Rendering the table of the SQL call output$tbl <- renderTable({ - # Using a memoised function allows to prevent from + # Using a memoised function prevents us from # calling the SQL database every time the user inputs # a change memoised_fct_sql(input$cut, con) @@ -324,7 +324,7 @@ The extra arguments you will find are `cacheKeyExpr` and `sizePolicy`: the forme The good news is that converting existing `renderPlot()` functions to `renderCachedPlot()` is pretty straightforward in most cases: take your current `renderPlot()`, and add the cache keys.[^optimizing-shiny-code-3] -[^optimizing-shiny-code-3]: In some cases you will have to configure the size policy, but in most cases the default values work just well. +[^optimizing-shiny-code-3]: In some cases you will have to configure the size policy, but in most cases the default values work just fine. Here is an example: @@ -354,7 +354,7 @@ server <- function( # Plotting the selected data.frame plot( get(input$tbl) ) }, cacheKeyExpr = { - # List here all the reactive expression that will + # List here all the reactive expressions that will # be used as cache key when running the app, # you will see that the first time you plot one # graph, it takes a couple of seconds, @@ -376,7 +376,7 @@ If you try this app, the first rendering of the three plots will take a little b library(shiny) ui <- function(){ tagList( - # Select a number of row to sample from mtcars + # Select a number of rows to sample from mtcars sliderInput( "nrows", "Number of rows", @@ -451,9 +451,9 @@ To launch code blocks in parallel, we will use a combination of two packages, `{ The first type of asynchronous programming in `{shiny}` **allows non-blocking programming in a cross-session context**. In other words, it is a programming method which is useful in the context of running one `{shiny}` session that is accessed by multiple users. -Natively, in `{shiny}`, if *user1* launches a 15-seconds computation, then *user2* has to wait for this computation to finish before launching their own 15-seconds computation, and *user3* has to wait the 15 seconds of *user1* plus the 15 seconds for user, etc. +Natively, in `{shiny}`, if *user1* launches a 15-second computation, then *user2* has to wait for this computation to finish before launching their own 15-second computation, and *user3* has to wait the 15 seconds of *user1* plus the 15 seconds for *user2*, etc. -With `{future}` and `{promises}`, each long computation is sent to be run somewhere else, so when *user1* launches their 15-seconds computation, they are not blocking the R process for *user2* and *user3*. +With `{future}` and `{promises}`, each long computation is sent to be run somewhere else, so when *user1* launches their 15-second computation, they are not blocking the R process for *user2* and *user3*. How does it work?[^optimizing-shiny-code-5] `{promises}` comes with two operators which will be useful in our case, `%...>%` and `%...!%`: the first being "what happens when the `future()` is solved?" (i.e. when the computation from the `future()` is completed), and the second is "what happens if the `future()` fails?" (i.e. what to do when the `future()` returns an error). @@ -465,7 +465,7 @@ Here is an example of using this skeleton: ```{r 16-optimizing-shiny-code-11, eval = FALSE} library(future) library(promises) -# We're opening several R session (future specific) +# We're opening several R sessions (future specific) plan(multisession) # We send our code to be run in another session future({ @@ -671,7 +671,7 @@ plan(multisession) ui <- function(){ tagList( - # This button trigger a future, we can click several times + # This button triggers a future, we can click several times # on it when the app is running actionButton("go", "go"), # This will receive the output of the future @@ -697,7 +697,7 @@ server <- function( observeEvent( input$go , { # When the user clicks on the button, the last_id - # is incremented of one + # is incremented by one rv$last_id <- rv$last_id + 1 last_id <- rv$last_id @@ -793,7 +793,7 @@ for (i in 1:5){ } Sys.sleep(10) # List the messages. As you can see, the entries in title -# are not in numerical order because they didn't came back +# are not in numerical order because they didn't come back # in the same order as they were sent list_messages(queue) ``` @@ -807,7 +807,7 @@ list_messages(queue) 5 5 5 READY ``` -For an example of an application built using `{promise}` and `{future}`, feel free to browse [engineering-shiny.org/shinyfuture/](https://engineering-shiny.org/shinyfuture/): there you will find an example of blocking and non-blocking processes. +For an example of an application built using `{promises}` and `{future}`, feel free to browse [engineering-shiny.org/shinyfuture/](https://engineering-shiny.org/shinyfuture/): there you will find an example of blocking and non-blocking processes. ```{r 16-optimizing-shiny-code-18, include = FALSE, error = TRUE} try(fs::dir_delete(tpd)) diff --git a/17-javascript.Rmd b/17-javascript.Rmd index 2222e92..e8657ce 100644 --- a/17-javascript.Rmd +++ b/17-javascript.Rmd @@ -9,7 +9,7 @@ knitr::opts_chunk$set( comment = "", eval = FALSE) Note you can build a successful, production-grade `{shiny}`[@R-shiny] application without ever writing a single line of JavaScript code. Even more when you can use a lot of tools that already bundle JavaScript functionalities: a great example of that being `{shinyjs}` [@R-shinyjs], which allows you to interact with your application using JavaScript, without writing a single line of JavaScript. -We chose to include this chapter in this book as it will help you get a better understanding on how `{shiny}` works at its core, and show you that getting at ease with JavaScript can help you get better at building web applications using R in the long run. +We chose to include this chapter in this book as it will help you get a better understanding of how `{shiny}` works at its core, and show you that getting at ease with JavaScript can help you get better at building web applications using R in the long run. It can also help you extend `{shiny}` with other JavaScript libraries, for example, using `{htmlwidgets}` [@R-htmlwidgets], when you get better at writing JavaScript. That being said, note also that every inclusion of external JavaScript code or library can present a security risk for your application, so don't include code you don't know/understand in your application unless you are sure of what you are doing. @@ -54,7 +54,7 @@ It's important to note here that the **communication happens in both directions* In fact, when we write a piece of code like `sliderInput("first_input", "Select a number", 1, 10, 5)`, what we are doing is creating a binding between JavaScript and R, where the JavaScript runtime (in the browser) listens to any event happening on the slider with the id `"plop"`, and whenever it detects that something happens to this element, something (most of the time its value) is sent back to R, and R does computation based on that value. With `output$bla <- renderPlot({})`, what we are doing is making the two communicate the other way around: we are telling JavaScript to listen to any incoming data from R for the `id` `"bla"`, and whenever JavaScript sees incoming data from R, it puts it into the proper HTML tag (here, JavaScript inserts the image received from R in the `<img>` tags with the id `bla`). -Even if everything is written in R, we **are** writing a web application, i.e.. +Even if everything is written in R, we **are** writing a web application, i.e. HTML, CSS and JavaScript elements. Once you have realized that, the possibilities are endless: in fact almost anything doable in a "classic" web app can be done in `{shiny}` with a little bit of tweaking. What this also implies is that getting (even a little bit) better at writing HTML, CSS, and especially JavaScript will make your app better, smaller, and more user-friendly, as JavaScript is a language that has been designed to interact with a web page: change element appearances, hide and show things, click somewhere, show alerts and prompts, etc. @@ -63,8 +63,8 @@ All these things are good examples of where you should be using JavaScript inste Moreover, the number of JavaScript libraries available on the web is tremendous; and the good news is that `{shiny}` has everything it needs to bundle external JavaScript libraries inside your application.[^javascript-2] -[^javascript-2]: This can also be done by wrapping a JS libraries inside a package, which will later be used inside an application. - See for example `{glouton}` [@R-glouton], which is a wrapper around the [`js-cookie` >https://github.com/js-cookie/js-cookie> JavaScript library. +[^javascript-2]: This can also be done by wrapping a JS library inside a package, which will later be used inside an application. + See for example `{glouton}` [@R-glouton], which is a wrapper around the [`js-cookie`](https://github.com/js-cookie/js-cookie) JavaScript library. This is what this section of the book aims at: giving you just enough JavaScript knowledge to lighten your `{shiny}` app, in order to improve the global user and developer experience. In this chapter, we will first review some JavaScript basics which can be used "client-side" only, i.e. only in your browser. @@ -86,7 +86,7 @@ This will open a new interface where you can have access to a JavaScript console Here, you can try your first JavaScript code! For example, you can try running `var message = "Hello world"; alert(message);`. -[^javascript-3]: You can now work with JavaScript in a server with Node.JS, but this won't be a useful software when working with `{shiny}`. +[^javascript-3]: You can now work with JavaScript in a server with Node.JS, but this won't be useful software when working with `{shiny}`. See linked resources to learn more. As you might have guessed, we will not be focusing on playing with JavaScript in your browser console: what we want to know is how to insert JavaScript code inside a `{shiny}` application. @@ -172,7 +172,7 @@ document.getElementsByTagName("div") Note that some of these methods have been introduced with ES6, which is a version of JavaScript that came out in 2015. This version of JavaScript is supported by most browsers since mid-2016 (and June 2017 for Firefox) (see [JavaScript Versions](https://www.w3schools.com/js/js_versions.asp) from W3Schools). Most of your users should now be using a browser version that is compatible with ES6, but that is something that you might want to keep in mind: browser version matters when it comes to using JavaScript. -Indeed, some companies (for internal reason) are still using old versions of Internet Explorer: a constraint you want to be aware of before starting to build the app, hence a question that you want to ask during the Design step. +Indeed, some companies (for internal reasons) are still using old versions of Internet Explorer: a constraint you want to be aware of before starting to build the app, hence a question that you want to ask during the Design step. ### About DOM events @@ -321,7 +321,7 @@ observeEvent(input$ok, { ```{r 17-javascript-4, eval = FALSE} output$ui <- renderUI({ - # Conditionnal rendering of the UI + # Conditional rendering of the UI if (this){ tags$p("First") } else { @@ -347,7 +347,7 @@ The `onclick` attribute can be added straight inside the HTML tag when possible: ```{r 17-javascript-5, eval = FALSE} # Building a button using the native HTML tag # (i.e. not using the actionButton() function) -# This button only goal is to launch this JS code +# This button's only goal is to launch this JS code # when it is clicked tags$button( "Show", @@ -358,8 +358,8 @@ tags$button( Or with `shiny::tagAppendAttributes()`: ```{r 17-javascript-6, eval = FALSE} -# Using tagAppendAttributes() allows to add attributes to the -# outputed UI element +# Using tagAppendAttributes() allows you to add attributes to the +# outputted UI element plotOutput( "plot" ) %>% tagAppendAttributes( @@ -547,7 +547,7 @@ For example, with: ```{js 17-javascript-16, eval = FALSE, echo = TRUE} // This function from the Shiny JavaScript object -// Allows to register an input name, and a value +// Allows you to register an input name, and a value Shiny.setInputValue("rand", Math.random()) ``` @@ -589,7 +589,7 @@ $(function(){ }); ``` -These events (getting the user name and the last plot clicked), can then be caught from the server side with: +These events (getting the user name and the last plot clicked) can then be caught from the server side with: ```{r 17-javascript-19, eval = FALSE} # We wait for the output of alertme(), which will set the @@ -674,7 +674,7 @@ mod_my_first_module_server <- function(input, output, session){ As said in the introduction to this chapter, running JavaScript code that you don't fully control/understand can be tricky and might open doors for external attacks. In many cases, for the most common JavaScript manipulations, it's safer to go for a package that has already been proved efficient: `{shinyjs}`. -This package, licensed in MIT since version 2.0.0, can be used to perform common JavaScript tasks: show, hide, alert, click, etc. +This package, licensed under MIT since version 2.0.0, can be used to perform common JavaScript tasks: show, hide, alert, click, etc. See [deanattali.com/shinyjs/](https://deanattali.com/shinyjs/) for more information about how to use this package. @@ -701,7 +701,7 @@ const get_rand_beer = () => { fetch("https://api.punkapi.com/v2/beers/random") // What do we do when we receive the data .then((data) =>{ - // TTurn the data to JSON + // Turn the data to JSON data.json().then((res) => { // Send the json to R Shiny.setInputValue("beer", res, {priority: 'event'}) @@ -732,7 +732,7 @@ Learn more about `fetch()` at [Using Fetch](https://developer.mozilla.org/en-US/ If you want to interact straight from R with NodeJS (JavaScript in the terminal), you can try the `{bubble}` [@R-bubble] package. Be aware that you will need to have a working NodeJS installation on your machine. -It can be installed from GitHub +It can be installed from GitHub: ```{r 17-javascript-23, eval = FALSE} remotes::install_github("ColinFay/bubble") diff --git a/18-css.Rmd b/18-css.Rmd index 6be1f9f..464dec3 100644 --- a/18-css.Rmd +++ b/18-css.Rmd @@ -13,7 +13,7 @@ But we might not be happy with what a "standard page" (with no CSS) looks like: If you want to get an idea of the importance of CSS, try installing extensions like [Web Developer](https://chrome.google.com/webstore/detail/web-developer/bfbameneiokkgbdmiekhjnmfkcnldhhm) for Google Chrome. Then, if you go on the extension and choose CSS, click "Disable All Style", to see what a page without CSS looks like. -For example, Figure \@ref(fig:18-css-1) is what [rtask.thinkr.fr](https://rtask.thinkr.fr) looks like with CSS, and Figure \@ref(fig:18-css-2) and Figure \@ref(fig:18-css-3) shows what it looks like without CSS. +For example, Figure \@ref(fig:18-css-1) is what [rtask.thinkr.fr](https://rtask.thinkr.fr) looks like with CSS, and Figure \@ref(fig:18-css-2) and Figure \@ref(fig:18-css-3) show what it looks like without CSS. (ref:rtaskcss) <https://rtask.thinkr.fr> with CSS. @@ -33,7 +33,7 @@ knitr::include_graphics("img/rtask_without_css.png") knitr::include_graphics("img/rtask_without_css2.png") ``` -CSS now seems pretty useful right? +CSS now seems pretty useful, right? ### `{shiny}`'s default: `fluidPage()` @@ -255,7 +255,7 @@ If you are building your application with `{golem}` [@R-golem], the good news is If you want to use an external CSS template, there are several packages that exist that can implement new custom UI designs for your application. Here are some: -- `{resume}`[@R-resume], provides an implementation of the [Bootstrap Resume Template](https://github.com/BlackrockDigital/startbootstrap-resume). +- `{resume}`[@R-resume] provides an implementation of the [Bootstrap Resume Template](https://github.com/BlackrockDigital/startbootstrap-resume). - `{nessy}` [@R-nessy], a port of [NES CSS](https://github.com/nostalgic-css/NES.css). diff --git a/19-appendix.Rmd b/19-appendix.Rmd index aa77143..4c0dff1 100644 --- a/19-appendix.Rmd +++ b/19-appendix.Rmd @@ -19,10 +19,10 @@ Hello! We want to build a small application that can minify CSS, JavaScript, HTML and JSON. -In this app, user will be able either to paste +In this app, users will be able either to paste the content or to upload a file. -Once the content is pasted/upladed, they select +Once the content is pasted/uploaded, they select the type, which is pre-selected based on the file extension. Then they click on a button, and the content is minified. @@ -49,7 +49,7 @@ Cheers! - The user might get different results based on the minifying algorithm they use, which can be surprising at first. The application should alert about this. -- For long printed outputs, if we use `verbatimTextOuput`, we should be careful about the page width, as these elements will natively overflow on the x-axis of the page. +- For long printed outputs, if we use `verbatimTextOutput`, we should be careful about the page width, as these elements will natively overflow on the x-axis of the page. This should be doable with the following CSS: `pre{ white-space: pre-wrap; word-break: keep-all; }`. - We should be careful about using semantic HTML for the inputs and outputs. @@ -125,7 +125,7 @@ company `r nms[1]` is a `{shiny}` developer at `r company[1]`. She's been learning R in graduate school while working on her master's degree in statistics. When she started at `r company[1]`, she was mainly doing data analysis in Rmd, but has gradually switched to building `{shiny}` applications full-time. -She discovered minification while reading the "Engineering Production-Grade Shiny App" book, and now wants to add this to her `{shiny}` application. +She discovered minification while reading the "Engineering Production-Grade Shiny Apps" book, and now wants to add this to her `{shiny}` application. #### `r nms[2]`: web developer and trainer at `r company[2]` {.unnumbered} @@ -152,7 +152,7 @@ minif <- paste0("`", minif, "()`") minif <- knitr::combine_words(minif) ``` -- `guess_minifier`, which will take a function and return the available algorithms for that file: for example, if you have a JavaScript file, you'll be able to use the `r minif` functions. If the type is not guessed based on the extension, the function should fail gracefully, and not make `{shiny}` crash. We'll chose to return an empty string if this extension is not guessed. +- `guess_minifier`, which will take a function and return the available algorithms for that file: for example, if you have a JavaScript file, you'll be able to use the `r minif` functions. If the type is not guessed based on the extension, the function should fail gracefully, and not make `{shiny}` crash. We'll choose to return an empty string if this extension is not guessed. ```{r 19-appendix-4} library(minifyr) @@ -161,8 +161,8 @@ guess_minifier <- function(file){ ext <- tools::file_ext(file) # Check that the extension is correct, if not, return early # It's important to do this kind of check also - # on the server side as HTML manual tempering - # would allow to also send other type of files + # on the server side as HTML manual tampering + # would allow one to also send other types of files if ( ! ext %in% c("js", "css", "html", "json") ){ @@ -211,7 +211,7 @@ guess_minifier( guess_minifier( minifyr_example("json") )[2:4] -# Try with a non valid extension +# Try with an invalid extension guess_minifier( "path/to/text.docx" ) @@ -361,7 +361,7 @@ Here are the few steps we will be working on: To do so, we will use `triggers` from `{gargoyle}`. - Chances are, users will be testing several algorithms when using the application, and we don't want the minification process to happen another time when it is called on the same file and with the same algorithm. - This is even even more important because the process involves calling an external NodeJS process. + This is even more important because the process involves calling an external NodeJS process. To prevent that, we will be caching the function that does the computation. - Create an unseen input that will upload data, so that we can build an interactivity test using `{crrry}`. @@ -423,7 +423,7 @@ As an example, we will deploy this app in three media: as a package, on RStudio ### Before deployment checklist {.unnumbered} -- [x] *`devtools::check()`, run from the command line, return 0 errors, 0 warnings, 0 notes* +- [x] *`devtools::check()`, run from the command line, returns 0 errors, 0 warnings, 0 notes* - [x] *The current version number is valid, i.e* if the current app is an update, the version number has been bumped: it makes sense, before the first deployment, to keep a version number of `0.0.0.9000`, and increment this dev version whenever we implement changes or do test deployments. Because this is a "true" deployment, we bumped the version to `0.1.0`. @@ -432,9 +432,9 @@ As an example, we will deploy this app in three media: as a package, on RStudio - [x] *Test coverage is good, i.e. you cover a sufficient amount of the codebase, and these tests cover the core/strategic algorithms*. -- [x] *The person to call if something goes wrong is clear to everyone involved in the product.*. +- [x] *The person to call if something goes wrong is clear to everyone involved in the product.* -- [x] *The debugging process is clear to everyone involved in the project, including how to communicate bugs to the developer team, and how long it will take to get changes implemented*: this project will be made open source, so the bug will have to be listed on the Github repo. +- [x] *The debugging process is clear to everyone involved in the project, including how to communicate bugs to the developer team, and how long it will take to get changes implemented*: this project will be made open source, so the bug will have to be listed on the GitHub repo. To help that, we added a link to the GitHub repository on the application. - [x] *(If relevant) The server it is deployed on has all the necessary software installed (Docker, Connect, `Shiny Server`, etc.) to make the application run*. @@ -453,7 +453,7 @@ As an example, we will deploy this app in three media: as a package, on RStudio - [x] *(If relevant) The server where the app is deployed has access to the data sources (database, API...)*: not relevant. -- [x] *If the app records data and there are backups for these data*: not relevant +- [x] *If the app records data and there are backups for these data*: not relevant. ### Deploy as a tar.gz {.unnumbered} @@ -461,7 +461,7 @@ To share an application as a tar.gz, you can call `devtools::build()`, which wil You can then share this archive, and install it with `remotes::install_local("path/to/tar.gz")`. Note that this can also be done with base R, but `{remotes}` offers a smarter way when it comes to managing the dependencies of your archived package. -This `tar.gz` can also be sent to a package repository; be it the CRAN or any other package manager you might have in your company. +This `tar.gz` can also be sent to a package repository; be it CRAN or any other package manager you might have in your company. ### Deploy on RStudio Connect {.unnumbered} @@ -477,7 +477,7 @@ Once this is done, we will create/update the `.dockerignore` file at the root of Removing the installation of `libgit2-dev` solved the issue. Inside our `Dockerfile`, we will also change the default repo to use <https://packagemanager.rstudio.com/all/latest>, which proposes precompiled packages for our system, making the installation faster. -We will also add an installation of NodeJS, which is needed by our application.. +We will also add an installation of NodeJS, which is needed by our application. Then, we can go to our terminal and compile the image! diff --git a/chapter-abstracts.Rmd b/chapter-abstracts.Rmd index 7c9bb49..8336040 100644 --- a/chapter-abstracts.Rmd +++ b/chapter-abstracts.Rmd @@ -11,7 +11,7 @@ knitr::opts_chunk$set(echo = TRUE) ## Chapter Author -- Colin Fay -## Chapter Number & Title -- Chapter 1, Chapter 1 About Successful {shiny} Apps +## Chapter Number & Title -- Chapter 1, About Successful {shiny} Apps The first chapter of the book will give the reader a short introduction to the `{shiny}` package. It will then lay the foundations for the rest of the book by defining key concepts necessary for a clear understanding of the upcoming chapters. Here, we will address the following questions: how can we define complexity when it comes to software engineering? @@ -26,7 +26,7 @@ How can we measure both these aspects when it comes to R and `{shiny}`? This second chapter of the book will cover a crucial concept when it comes to leading a successful software engineering project: planning. -In this chapter, the reader will be presented project management and planning in the context of a `{shiny}` project: why it's important to plan ahead, how to leverage the KISS principle, and how to practically organize a team of `{shiny}` developers, both from the tooling and management point of views. +In this chapter, the reader will be presented with project management and planning in the context of a `{shiny}` project: why it's important to plan ahead, how to leverage the KISS principle, and how to practically organize a team of `{shiny}` developers, both from the tooling and management points of view. ## Book Title -- Engineering Production-Grade Shiny Apps @@ -44,9 +44,9 @@ In this chapter, we will cover the importance of building a `{shiny}` applicatio ## Chapter Number & Title – Chapter 4, Introduction to {golem} -Chapter will introduce `{golem}`, a framework for building production-grade `{shiny}` applications. +This chapter will introduce `{golem}`, a framework for building production-grade `{shiny}` applications. -In this chapter, we will give an introduction to the general philosophy behind `{golem}`, and will describe in details the structure of a `{golem}` project: what makes it similar to a standard R package (`DESCRIPTION`, `NAMESPACE`, `man/`, ...), and what makes it unique. Notably, we will spend some time reviewing the functions contained in the default `{golem}` project, the very same functions that will be the starting point of your future `{shiny}` applications. +In this chapter, we will give an introduction to the general philosophy behind `{golem}`, and will describe in detail the structure of a `{golem}` project: what makes it similar to a standard R package (`DESCRIPTION`, `NAMESPACE`, `man/`, ...), and what makes it unique. Notably, we will spend some time reviewing the functions contained in the default `{golem}` project, the very same functions that will be the starting point of your future `{shiny}` applications. ## Book Title -- Engineering Production-Grade Shiny Apps @@ -62,13 +62,13 @@ This fifth chapter succinctly presents the steps of the workflow later developed ## Chapter Number & Title – Chapter 6, UX Matters -Chapter 6 covers the basics of what make a web application user-friendly. +Chapter 6 covers the basics of what makes a web application user-friendly. -First, we cover the importance of simplicity when it comes to designing an interface, keeping in mind the "Don't make me think" mantra. When reading the web, users tend to scan, instead of cautiously make logical decisions about how to behave. In other word, they do not really read but tend to scan the content, making it crucial for the page to be as simple as possible, so that the visitors easily find their way through the interface. +First, we cover the importance of simplicity when it comes to designing an interface, keeping in mind the "Don't make me think" mantra. When reading the web, users tend to scan, instead of cautiously making logical decisions about how to behave. In other words, they do not really read but tend to scan the content, making it crucial for the page to be as simple as possible, so that the visitors easily find their way through the interface. -Then, we cover the importance of lowering complexity of a software by restraining from implementing way too many features: when building an application with `{shiny}`, it's crucial to think about the necessity of the features we're implementing, so that we don't end with too much reactivity, and/or an application that is way to slow to be used. +Then, we cover the importance of lowering complexity of software by refraining from implementing way too many features: when building an application with `{shiny}`, it's crucial to think about the necessity of the features we're implementing, so that we don't end up with too much reactivity, and/or an application that is way too slow to be used. -Finally, we cover one of the most important topic when it comes to making a successful application: accessibility. In other words, how do we make an application usable by the widest audience possible? How do we work on making our application usable by people with visual, mobility, or cognitive disabilities. This chapter will introduce the notion of semantic HTML in the context of `{shiny}`, structure, and, something important in a context where we build applications with data visualization: choice of colors. +Finally, we cover one of the most important topics when it comes to making a successful application: accessibility. In other words, how do we make an application usable by the widest audience possible? How do we work on making our application usable by people with visual, mobility, or cognitive disabilities? This chapter will introduce the notion of semantic HTML in the context of `{shiny}`, structure, and, something important in a context where we build applications with data visualization: choice of colors. ## Book Title -- Engineering Production-Grade Shiny Apps @@ -76,9 +76,9 @@ Finally, we cover one of the most important topic when it comes to making a succ ## Chapter Number & Title -- Chapter 7, -This seventh chapter covers the need for getting yourself prepared upfront when it comes to engineering a `{shiny}`. +This seventh chapter covers the need for getting yourself prepared upfront when it comes to engineering a `{shiny}` application. -Here, we will cover the importance of starting with planning, thinking, and evaluating existing solution before rushing into coding. We will also introduce concept maps, and give a series of tools to evaluate the project before even writing a single line of code. In other words, we'll see how to get started with user interview, how to create personas, and why it's important to evaluate pre-existing codebase before starting the project. +Here, we will cover the importance of starting with planning, thinking, and evaluating existing solutions before rushing into coding. We will also introduce concept maps, and give a series of tools to evaluate the project before even writing a single line of code. In other words, we'll see how to get started with user interviews, how to create personas, and why it's important to evaluate the pre-existing codebase before starting the project. ## Book Title – Engineering Production-Grade Shiny Apps @@ -119,9 +119,9 @@ Inside this chapter will be described how to add external dependencies to your a ## Chapter Number & Title -- Chapter 11, Build yourself a safety net -The fourth part of the workflow is described in chapter 11 and 12. +The fourth part of the workflow is described in chapters 11 and 12. -In this chapter 11, we come back in more details on the importance of testing your application, and on the toolkit `{shiny}` developers can leverage to test their applications: `{testthat}` for the business logic, `puppeteer`, `{shinytest}` and `{crrry}` for the front-end, and `{shinyloadtest}` and `{dockerstats}` for the application load, i.e. the computer resources necessary to make your application work. +In this chapter 11, we come back in more detail on the importance of testing your application, and on the toolkit `{shiny}` developers can leverage to test their applications: `{testthat}` for the business logic, `puppeteer`, `{shinytest}` and `{crrry}` for the front-end, and `{shinyloadtest}` and `{dockerstats}` for the application load, i.e. the computer resources necessary to make your application work. In this chapter, we will also present two tools that are crucial to reproducibility: `{renv}`, a package to manage local dependencies at a project level, and `Docker`, one of the most popular software today to write and deploy software. @@ -134,9 +134,9 @@ In this chapter, we will also present two tools that are crucial to reproducibil Chapter 12 covers another essential part of working on solid grounds when building a production-grade software: version control. -Version control is a methodology, based on a software, that allows to track change of a software through time, and to work simultaneously on various versions of the same codebase without interference. In this chapter, we will be focusing on `git`, one of the most popular version control system, and present a methodology called 'git flow'. +Version control is a methodology, based on software, that allows tracking changes of software through time, and to work simultaneously on various versions of the same codebase without interference. In this chapter, we will be focusing on `git`, one of the most popular version control systems, and present a methodology called 'git flow'. -This chapter will also cover how you can use version control server like GitLab or GitHub to build Continuous Integration and Continous Deployment (CI/CD) processes, allowing to automate actions whenever new content is integrated to the main codebase. +This chapter will also cover how you can use version control servers like GitLab or GitHub to build Continuous Integration and Continuous Deployment (CI/CD) processes, allowing you to automate actions whenever new content is integrated into the main codebase. ## Book Title -- Engineering Production-Grade Shiny Apps @@ -146,7 +146,7 @@ This chapter will also cover how you can use version control server like GitLab Chapter 13 covers the last part of the workflow: deployment. -This chapter starts with a checklist of things to do before sending an application to production. Then, we move to the three main ways to share a `{shiny}` application: as a package, via a package manager or by sharing a `tar.gz`, using one of RStudio deployment platforms, or finally how you can leverage Docker to deploy your application to production. +This chapter starts with a checklist of things to do before sending an application to production. Then, we move to the three main ways to share a `{shiny}` application: as a package, via a package manager or by sharing a `tar.gz`, using one of RStudio's deployment platforms, or finally how you can leverage Docker to deploy your application to production. ## Book Title – Engineering Production-Grade Shiny Apps @@ -156,7 +156,7 @@ This chapter starts with a checklist of things to do before sending an applicati Chapter 14 starts with the reflection around the necessity to optimize, and around managing the optimization process when building your application. -When building a `{shiny}` application that will be send to production, schedule matters, and focusing on optimizing too soon, or too much, might endanger the whole success of the project. On the other hand, choices made when optimizing the application might have a big impact on the longevity of the project. +When building a `{shiny}` application that will be sent to production, schedule matters, and focusing on optimizing too soon, or too much, might endanger the whole success of the project. On the other hand, choices made when optimizing the application might have a big impact on the longevity of the project. If you decide to go along the optimization road, you better start by benchmarking what and where you need to optimize, so that you're sure you're not working on optimizing parts of the application that do not need to be optimized. The second part of the chapter presents tools you can use to perform this code profiling. @@ -166,7 +166,7 @@ If you decide to go along the optimization road, you better start by benchmarkin ## Chapter Number & Title -- Chapter 15, Common Application Caveats -Some code bottlenecks (parts of the codebase that slow the application) might not be caught by simple profiling tools: sometime a slow application can be explained by caveats in the way the application is designed. +Some code bottlenecks (parts of the codebase that slow the application) might not be caught by simple profiling tools: sometimes a slow application can be explained by caveats in the way the application is designed. This chapter will present three main sources of design patterns that might be slowing your application: uncontrolled reactivity, where too much happens (also known as "reactivity hell"), making R perform too much computation, and data source management. @@ -179,7 +179,7 @@ This chapter will present three main sources of design patterns that might be sl There are several methods you can choose to optimize your application so that it works faster, and handles memory in a more efficient way. This chapter presents three of them. -First, focusing on the R code itself. Then, caching elements. Caching is the process of storing computation results from a function so that it can be reused, instead of recomputing the function every time. In this chapter, we will see how you can implement caching with R code, and how to use the `{shiny}`-specific functions to do that. Finally, we will present a way to build asynchronousity inside a `{shiny}` application using `{promises}` and `{future}`. +First, focusing on the R code itself. Then, caching elements. Caching is the process of storing computation results from a function so that it can be reused, instead of recomputing the function every time. In this chapter, we will see how you can implement caching with R code, and how to use the `{shiny}`-specific functions to do that. Finally, we will present a way to build asynchronicity inside a `{shiny}` application using `{promises}` and `{future}`. ## Book Title – Engineering Production-Grade Shiny Apps @@ -187,9 +187,9 @@ First, focusing on the R code itself. Then, caching elements. Caching is the pro ## Chapter Number & Title – Chapter 17, Using JavaScript -One of the best way to enhance your application, in the long run, is to get comfortable with JavaScript, a scripting language that runs in your web browser. +One of the best ways to enhance your application, in the long run, is to get comfortable with JavaScript, a scripting language that runs in your web browser. -With JavaScript, you'll be able to enhance the user experience by leveraging in-browser events, and lower the computation performed by R as you can perform them inside the browser, leaving more space to the server for computation. And in the long run, when you're comfortable with JavaScript, a whole world of `{shiny}` extensions opens. +With JavaScript, you'll be able to enhance the user experience by leveraging in-browser events, and lower the computation performed by R as you can perform it inside the browser, leaving more space to the server for computation. And in the long run, when you're comfortable with JavaScript, a whole world of `{shiny}` extensions opens. But in the meantime, you might be looking for a place to start. Search no more, this chapter is the perfect place to get started with JavaScript for `{shiny}`. @@ -201,6 +201,6 @@ But in the meantime, you might be looking for a place to start. Search no more, In this last chapter of the book, we will introduce the last of the HTML/JavaScript/CSS web technology trio. -CSS, for Cascading StyleSheets, is what powers the design of the web page. When making `{shiny}` applications that will be sent to production, chances are that you'll want to enhance its design. Then, CSS is where you should start. +CSS, for Cascading Style Sheets, is what powers the design of the web page. When making `{shiny}` applications that will be sent to production, chances are that you'll want to enhance their design. Then, CSS is where you should start. This chapter gives a short introduction to this language, so that you're more comfortable tackling design tasks on your next application. diff --git a/index.Rmd b/index.Rmd index 3478c8b..e25a100 100644 --- a/index.Rmd +++ b/index.Rmd @@ -11,7 +11,7 @@ biblio-style: apalike link-citations: yes colorlinks: yes site: bookdown::bookdown_site -description: "A book about engineering shiny application that will later be sent to production. This book cover project management, structuring your project, building a solid testing suite, and optimizing your codebase. We describe in this book a specific workflow: design, prototype, build, strengthen and deploy." +description: "A book about engineering shiny applications that will later be sent to production. This book covers project management, structuring your project, building a solid testing suite, and optimizing your codebase. We describe in this book a specific workflow: design, prototype, build, strengthen and deploy." favicon: img/favicon.ico github-repo: ThinkR-open/engineering-shiny-book graphics: yes @@ -59,7 +59,7 @@ options(width = 55) # Introduction {-} -<img src="img/engineering-shiny.jpeg" width="300" align="right" alt="" class="cover" /> Welcome to "Engineering Production-Grade Shiny Apps" by [Colin Fay](https://colinfay.me/), [Sébastien Rochette](statnmap.com), [Vincent Guyader](https://rtask.thinkr.fr/consultants-r-experts/vincent-guyader/) and [Cervan Girard](https://rtask.thinkr.fr/consultants-r-experts/cervan-girard). +<img src="img/engineering-shiny.jpeg" width="300" align="right" alt="" class="cover" /> Welcome to "Engineering Production-Grade Shiny Apps" by [Colin Fay](https://colinfay.me/), [Sébastien Rochette](https://statnmap.com), [Vincent Guyader](https://rtask.thinkr.fr/consultants-r-experts/vincent-guyader/) and [Cervan Girard](https://rtask.thinkr.fr/consultants-r-experts/cervan-girard). `r if (knitr::is_html_output()) ' @@ -72,7 +72,7 @@ The online version of this book is free to read here (thanks to Chapman & Hall/C ## Motivation {-} -This book will not __get you started with `{shiny}`__, nor __talk how to work with `{shiny}` once it is sent to production__. +This book will not __get you started with `{shiny}`__, nor __talk about how to work with `{shiny}` once it is sent to production__. What we will be discussing in this book is __the process of building an application that will later be sent to production__. Why this topic? Lots of blog posts and books talk about getting started with `{shiny}` [@R-shiny] or about what to do once your application is ready to be sent to production. @@ -119,14 +119,14 @@ We like to think that a piece of software is in production when it combines the These three properties impact two specific groups: users and the developers. Indeed, the users rely on the app to work so that they can do their job, and expect it to deliver meaningful results that they can count on. -From the engineering point of view, a production-grade software can be relied upon in the sense that developers count on it to run as expected, and they need to rely on the software to be resilient to change, _i.e_ to be modular, documented, and strongly tested so that changes can be integrated with confidence. +From the engineering point of view, production-grade software can be relied upon in the sense that developers count on it to run as expected, and they need to rely on the software to be resilient to change, _i.e_ to be modular, documented, and strongly tested so that changes can be integrated with confidence. -A production software also has real-life impact if something goes wrong: users will make wrong decisions, they might be unable to do their day-to-day work, and there are all the things that can happen when the software you use on a daily basis fails to run. -From the engineering point of view, a production-grade software has real impact when something goes wrong: someone has to fix the bug, the company selling the software might lose money, data can be lost, and so on. +Production software also has real-life impact if something goes wrong: users will make wrong decisions, they might be unable to do their day-to-day work, and there are all the things that can happen when the software you use on a daily basis fails to run. +From the engineering point of view, production-grade software has real impact when something goes wrong: someone has to fix the bug, the company selling the software might lose money, data can be lost, and so on. Given these two properties, you can understand why being in production doesn't necessarily mean being served to gazillions of users,^[ "Production" being equal to tons of users is a definition we regularly hear. -] and serving trillions of gigabytes of data: even software that is used by one person who relies on this application to do their job is a production software. +] and serving trillions of gigabytes of data: even software that is used by one person who relies on this application to do their job is production software. This is what this book is about: building `{shiny}` applications that can be used, on which you and your users can rely, and including all the tools that will help you prevent things from going wrong, and when they eventually do, making sure you are equipped to quickly fix the bugs. @@ -134,9 +134,9 @@ This is what this book is about: building `{shiny}` applications that can be use + Part 1, "Building Successful `{shiny}` Apps" gives a __general context for what we mean by "production-grade" and "successful" `{shiny}` applications, and what challenges arise when you are dealing with a large-scale application designed for production__. In this part, we will define what we mean by "Successful", stress the importance of project management, develop how to structure your project for production, and introduce the `{golem}` [@R-golem] package. -We will finally, briefly introduce to our development workflow: a workflow that will be explored in Parts 2 to 6. +We will finally, briefly introduce our development workflow: a workflow that will be explored in Parts 2 to 6. -+ Part 2 through 6 __explore the workflow for building successful applications__. ++ Parts 2 through 6 __explore the workflow for building successful applications__. _Part 2: Design_ (Step 1) underlines the centrality of the user experience when engineering an application, and emphasizes the importance of designing before coding. _Part 3: Prototype_ (Step 2) stresses the importance of prototyping, explores the setting of a `{golem}`-based application, and presents `{shinipsum}`, `{fakir}`, and the "Rmd First" development methodology. _Part 4: Build_ (Step 3) explores the building step of the application, _i.e_ the core engineering of the application once the prototyping phase is finished. @@ -181,7 +181,7 @@ He strongly believes that meeting highly technical challenges is not incompatibl Cervan has worked on some of the example applications that are used inside this book, namely `{shinipsumdemo}`, `{databasedemo}`, `{grayscale}`, `{bs4dashdemo}`, and `{shinyfuture}`. Cervan is a Data Scientist at ThinkR. -He is enthusiastic and motivated when it comes to rolling up his sleeves for new challenges, even if it means venturing dangerously into the depths of R, learning new languages, and experimenting outside your comfort zone. +He is enthusiastic and motivated when it comes to rolling up his sleeves for new challenges, even if it means venturing dangerously into the depths of R, learning new languages, and experimenting outside his comfort zone. Whatever the challenge, he remains reliable, constructive, and efficient when it comes to using his skills to train or develop. He also enjoys training learners of all levels in the R language. @@ -203,7 +203,7 @@ This book was built with `{knitr}` [@R-knitr] and `{bookdown}` [@R-bookdown]. Package names are in curly brackets in code format (e.g., `{rmarkdown}`), and inline code and file names are formatted in a typewriter font (e.g., `knitr::knit('doc.Rmd')`). Function names are formatted in a typewriter font and followed by parentheses (e.g., `render_book()`). -Larger code blocks are formatted in a typewriter font and have a gray backgroud, e.g.: +Larger code blocks are formatted in a typewriter font and have a gray background, e.g.: ```{r index-2, eval=FALSE} install.packages("golem")