Communicate with other types of APIs

️✅ Learning objectives

• Fetch data from GraphQL APIs.
• Fetch data from websocket APIs.
• Fetch data from gRPC APIs.

library(ghql)
library(jsonlite)
library(dplyr)
library(httr2)
library(websocket)
library(protolite)
# library(RProtoBuf)

GraphQL

What is GraphQL?

• “Open-source data query and manipulation language”
  • query = data you need
  • mutation = new data to add
  • JSON-like format
  • 1 endpoint
• Developed by Facebook
• Used by Facebook, GitHub, Yelp, Shopify, etc

• Think of it as a REST API with 1 endpoint and strict rules about what goes into the request body.
• Might get a chapter in my next iteration.
• Intended to allow you to filter for the data you need.
  • Could really use a dplyr implementation!

GraphQL in R

• Client side: {ghql}
• Server side: {gqlr}

• Current landscape feels like “I use GraphQL in other languages, and want it to technically be possible to do so from R.”
• We’ll go through a {ghql} example

Country data: setup

• Countries GraphQL API

ghql_con <- ghql::GraphqlClient$new(url = "https://countries.trevorblades.com/")
qry <- ghql::Query$new()

• UpperCamelCase + $new() means this uses R6.
  • See Advanced R to get started

Country data: query

qry$query("country_data",
  "query($code: [String!]) {
     countries(
       filter: {code: {in: $code}}
     ) {
       code
       name
       capital
       phone
       languages {
         code
         name
       }
     }
   }"
)

# name of this query
# define variables
#
#
# Select countries using codes
#
# fields we want for each country

• No commas
• No quotation marks

• “[String!]” means “Array of non-nullable (required) strings”.
• Schema specifies data types
• Could filter by more things (like dplyr filter)
  • Schema specifies code, continent, currency, name

GraphQL response

country_codes <- list(code = c("US", "DE"))
x <- ghql_con$exec(
  qry$queries$country_data,
  country_codes
) # We created a query named "country_data"
jsonlite::fromJSON(x)
#> $data
#> $data$countries
#>   code          name         capital phone   languages
#> 1   DE       Germany          Berlin    49  de, German
#> 2   US United States Washington D.C.     1 en, English

websocket

What is websocket?

• Alternative to HTTP
• ws:// or wss:// (vs http:// / https://)
• 2-way communication
• {websocket} package

• Package by posit.
• Used by {chromote}, {shinyloadtest}.
• (next slide for Shiny)

Aside: shiny & websockets

• Sometimes see websocket-related errors from Shiny
• Shiny UI & server communicate via a websocket connection
• Doesn’t use {websocket}

• More directly managed via {httpuv} & TypeScript code
• Details of how this works are beyond the scope of this book.

websocket usecases

• News/Message feeds
  • Display new content as it comes in
• Messaging
  • Send and receive ~simultaneously without new connections
• Multi-player games
• Collaborative editing
• Real-time dashboards

• But not good for reproducible analyses, so you probably won’t use it
• Do people WANT to see the demo to see how it works?

websocket demo: setup

ws <- websocket::WebSocket$new("ws://echo.websocket.events/", autoConnect = FALSE)
ws$onMessage(\(event) {
  now <- format(Sys.time(), digits = 0)
  cat("Client got msg:", event$data, "at", now, "\n")
})

websocket demo

ws$connect()
#> Client got msg: echo.websocket.events sponsored by Lob.com at 2023-11-15 2023-11-15 08:28:04
# (can do other things in console now)
1
#> [1] 1
ws$send("hello")
#> Client got msg: hello at 2023-11-15 08:28:10
ws$close()

(switch to RStudio)

websocket: toward usefulness

ws_counter <- 1
ws2 <- websocket::WebSocket$new(
  "ws://echo.websocket.events/", autoConnect = FALSE
)
ws2$onMessage(\(event) {
  ws_counter <<- ws_counter + 1 # Add 1 to global ws_counter var 
  cat(ws_counter, "\n")
})
ws_counter
#> 1
ws2$connect()
#> 2
ws2$send("update")
#> 3
ws2$send("update again")
#> 4
ws2$close()

• Load Environment tab in RStudio
• I’ve done this with a Slack connection, where a variable always holds the latest message (for example)
  • But haven’t actually USED that.

gRPC

What is gRPC?

• “Google Remote Procedure Call”
• Becoming very popular
• Good for real-time, 2-way communication
• Uses HTTP/2 (upgraded HTTP, more socket-like)
  • Can access via {httr2}
• “Protocol buffers” datatype
  • Fully implemented in {RProtoBuf} package
  • {protolite} for basics

• “Remote procedure call” means “call a function on another computer”
• Often used as a way to have different functions distributed to specialized machines
• Used primarily for tightly-coupled APIs (“microservices”)
• Datatype transfers fast but also turns back into native in-memory form fast

gRPC demo

From RProtoBuf paper

resp <- httr2::request("https://demo.ocpu.io/MASS/data/Animals/pb") |> 
  httr2::req_perform()
output <- httr2::resp_body_raw(resp) |> 
  protolite::unserialize_pb() # This is the important part
identical(output, MASS::Animals)
#> [1] TRUE
head(output)
#>                     body brain
#> Mountain beaver     1.35   8.1
#> Cow               465.00 423.0
#> Grey wolf          36.33 119.5
#> Goat               27.66 115.0
#> Guinea pig          1.04   5.5
#> Dipliodocus     11700.00  50.0

• “Serialize” is computer-science for “save”, or technically “convert to the format that’s needed in order to transfer it to this other thing”

gRPC demo2: R via API

pkg <- "tibble"
func <- "tibble"
args <- list(id = 1:3, letter = sample(letters, 3))
payload <- protolite::serialize_pb(args) # To protobuff
resp <- httr2::request("https://cloud.opencpu.org/ocpu/library") |> 
  httr2::req_url_path_append(pkg, "R", func, "pb") |> 
  httr2::req_body_raw(payload, type = "application/protobuf") |>
  httr2::req_perform()
httr2::resp_body_raw(resp) |> 
  protolite::unserialize_pb() # Back to R
#> # A tibble: 3 × 2
#>      id letter
#> * <int> <chr>
#> 1     1 n
#> 2     2 f
#> 3     3 k

• 88 packages, including some of the tidyverse packages (but not tidyverse itself, and not dplyr)
• I don’t have a use-case for this YET.
  • I might do something like this for DSLC microservices
• Seems to cache results server-side, not sure how often it clears