Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Sign up
Appearance settings

Commit 375fcb5

Browse files
[feat] Generate URL of wget download script using ESGF RESTful API (#74)
1 parent cc62c23 commit 375fcb5

24 files changed

+132
-87
lines changed

‎DESCRIPTION‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
Package: epwshiftr
22
Title: Create Future 'EnergyPlus' Weather Files using 'CMIP6' Data
3-
Version: 0.1.3.9012
3+
Version: 0.1.3.9013
44
Authors@R: c(
55
person(given = "Hongyuan",
66
family = "Jia",

‎NEWS.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -82,8 +82,8 @@
8282
* Now HDF5 format is supported (#60).
8383
* Now `replica` can be `NULL` in `esgf_query()` and `init_cmip6_index()`. In
8484
this case, both the master record and replicas are all returned (#61).
85-
* New class `EsgfQuery` is added to support more flexible. Please see
86-
`?EsgfQuery` for details (#63).
85+
* New class `EsgfQuery` is added to support more flexible APIs for ESGF search
86+
services. Please see `?EsgfQuery` for details (#63, #69).
8787

8888
## Bug fixes
8989

‎R/dict.R‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
#' CMIP6 Controlled Vocabularies (CVs) and Data Request Dictionary
22
#'
3-
#' The `Cmip6Dict` object provides functionalities to fetch the latested CMIP6
3+
#' The `Cmip6Dict` object provides functionalities to fetch the latest CMIP6
44
#' Controlled Vocabularies (CVs) and Data Request (DReq) information.
55
#'
66
#' The CMIP6 CVs gives a well-defined set of global attributes that are recorded

‎R/epwshiftr-package.R‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
#' i.e. monthly-mean fractional change, when performing morphing operations.
1515
#' The default value is set to `3`. If the morphing methods are set
1616
#' `"stretch"` or `"combined"`, and the absolute alpha exceeds the threshold
17-
#' value, warnings are issued and the morphing method is fallbacked to
17+
#' value, warnings are issued and the morphing method fallbacks to
1818
#' `"shift"` to avoid unrealistic morphed values.
1919
#' * `epwshiftr.dir`: The directory to store package data, including CMIP6
2020
#' model output file index and etc. If not set, the current user data

‎R/esgf.R‎

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -24,11 +24,10 @@ RES_FILE <- c(
2424
#'
2525
#' The ESGF search service exposes a RESTful URL that can be used by clients to
2626
#' query the contents of the underlying search index, and return results
27-
#' matching the given constraints. ributed capabilities of the ESGF search, the
28-
#' URL at any Index Node can be used to query that Node only, or all Nodes in
29-
#' the ESGF system. `esgf_query()` uses the [LLNL (Lawrence Livermore National
30-
#' Laboratory)](http://esgf-node.llnl.gov)
31-
#' Index Node.
27+
#' matching the given constraints. With the distributed capabilities of the ESGF
28+
#' search, the URL at any Index Node can be used to query that Node only, or all
29+
#' Nodes in the ESGF system. `esgf_query()` uses the
30+
#' [LLNL (Lawrence Livermore National Laboratory) Index Node](http://esgf-node.llnl.gov).
3231
#'
3332
#' The core Controlled Vocabularies (CVs) for use in CMIP6, including all
3433
#' activities, experiment, sources (GCMs), frequencies can be found at the
@@ -76,7 +75,7 @@ RES_FILE <- c(
7675
#' * `hursmin`: Minimum near-surface relative humidity, units: `%`.
7776
#' * `psl`: Sea level pressure, units: `Pa`.
7877
#' * `rsds`: Surface downwelling shortwave radiation, units: `W m-2`.
79-
#' * `rlds`: Surface downwelling longware radiation, units: `W m-2`.
78+
#' * `rlds`: Surface downwelling longwave radiation, units: `W m-2`.
8079
#' * `sfcWind`: Near-surface (usually, 10 meters) wind speed, units: `m s-1`.
8180
#' * `pr`: Precipitation, units: `kg m-2 s-1`.
8281
#' * `clt`: Total cloud area fraction for the whole atmospheric column, as

‎R/morph.R‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -237,8 +237,8 @@ remove_units <- function (data, var) {
237237
#' | 15 | `hour` | Integer | The hour of the morphed data |
238238
#' | 16 | `minute` | Integer | The minute of the morphed data |
239239
#' | 17 | **Variable Name** | Double | The morphed data, where `Variable Name` is the corresponding EPW weather variable name |
240-
#' | 18 | `delta` | Double | The shift factor. Will be `NA` for derivied values |
241-
#' | 19 | `alpha` | Double | The stretch factor. Will be `NA` for derivied values |
240+
#' | 18 | `delta` | Double | The shift factor. Will be `NA` for derived values |
241+
#' | 19 | `alpha` | Double | The stretch factor. Will be `NA` for derived values |
242242
#'
243243
#' @references
244244
#' Belcher, S., Hacker, J., Powell, D., 2005. Constructing design weather data
@@ -322,7 +322,7 @@ morphing_epw <- function (data, years = NULL, labels = NULL, methods = NULL, war
322322
)]
323323
rlds <- data_cmip[J("rlds"), on = "variable", nomatch = NULL]
324324
if (!nrow(rlds)) {
325-
verbose("WARNING: Input does not contain any data of 'surface downwelling longware radiation'. Skip.")
325+
verbose("WARNING: Input does not contain any data of 'surface downwelling longwave radiation'. Skip.")
326326
hor_ir <- data.table()
327327
} else {
328328
hor_ir <- morphing_hor_ir(data_epw, rlds, years, labels = labels, type = methods["hor_ir"], warning = warning)
@@ -849,7 +849,7 @@ solar_angle <- function (latitude, longitude, day_of_year, hour, timezone) {
849849
#'
850850
#' @param morphed An `epw_cmip6_morphed` object created using [morphing_epw()].
851851
#' @param by A character vector of columns to be used as grouping variables when
852-
#' creating EPW files. Should be a subeset of:
852+
#' creating EPW files. Should be a subset of:
853853
#'
854854
#' * `"experiment"`: root experiment identifiers
855855
#' * `"source"`: model identifiers

‎R/netcdf.R‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,7 @@ get_nc_meta <- function (file) {
5454
#' output file index. So the names of NetCDF files do not necessarily follow the
5555
#' CMIP6 file name encoding.
5656
#'
57-
#' `summary_database()` will append 5 columns in the CMIP6 ouput file index:
57+
#' `summary_database()` will append 5 columns in the CMIP6 output file index:
5858
#'
5959
#' - `file_path`: the full path of matched NetCDF file for every case.
6060
#'
@@ -64,11 +64,11 @@ get_nc_meta <- function (file) {
6464
#' `summary_database()` uses `future::sequential` backend, which runs things in
6565
#' sequential.
6666
#'
67-
#' @param dir A single string indcating the directory where CMIP6 model output
67+
#' @param dir A single string indicating the directory where CMIP6 model output
6868
#' NetCDF files are stored.
6969
#'
7070
#' @param by The grouping column to summary the database status. Should be a
71-
#' subeset of:
71+
#' subset of:
7272
#'
7373
#' * `"experiment"`: root experiment identifiers
7474
#' * `"source"`: model identifiers
@@ -91,7 +91,7 @@ get_nc_meta <- function (file) {
9191
#' when running current summary. Only applicable when `append` is set to
9292
#' `TRUE`. If `"keep"`, the metadata for the missing output files will
9393
#' be kept. If `"overwrite"`, existing metadata of those output will be
94-
#' first removed from the output file index and overwriten based on the
94+
#' first removed from the output file index and overwritten based on the
9595
#' newly matched files if possible. Default: `"keep"`.
9696
#'
9797
#' @param recursive If `TRUE`, scan recursively into directories. Default:
@@ -253,7 +253,7 @@ summary_database <- function (
253253
} else if (miss == "overwrite") {
254254
paste(
255255
"Since `miss` is set to `\"overwrite\"`, its metadata has been **removed** from the output file index",
256-
"and will be overwriten by the data from new matched NetCDF file if possible.\n"
256+
"and will be overwritten by the data from new matched NetCDF file if possible.\n"
257257
)
258258
},
259259
paste0(mes, collapse = "\n"),
@@ -605,10 +605,10 @@ get_nc_data <- function (x, coord, years, unit = TRUE) {
605605
#'
606606
#' @return An `epw_cmip6_data` object, which is basically a list of 3 elements:
607607
#'
608-
#' * `epw`: An [eplusr::Epw] object whose longitude and latitute are used to
608+
#' * `epw`: An [eplusr::Epw] object whose longitude and latitude are used to
609609
#' extract CMIP6 data. It is the same object as created in [match_coord()]
610610
#' * `meta`: A list containing basic metadata of input EPW, including `city`,
611-
#' `state_province`, `country`, `latitute` and `longitude`.
611+
#' `state_province`, `country`, `latitude` and `longitude`.
612612
#' * `data`: An empty [data.table::data.table()] if `keep` is `FALSE` or a
613613
#' [data.table::data.table()] of 14 columns if `keep` is `TRUE`:
614614
#'

‎R/query.R‎

Lines changed: 28 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -180,16 +180,17 @@ format.EsgfQueryParam <- function(x, encode = TRUE, space = FALSE, ...) {
180180
#' input for parameter setters.
181181
#'
182182
#' - \href{#method-EsgfQuery-url}{\code{EsgfQuery$url()}}: Returns the actual
183-
#' query URL.
183+
#' query URL or the wget script URL which can be used to download all files
184+
#' matching the given constraints..
184185
#'
185186
#' - \href{#method-EsgfQuery-response}{\code{EsgfQuery$response()}}: Returns the
186-
#' acutal response of
187+
#' actual response of
187188
#' \href{#method-EsgfQuery-count}{\code{EsgfQuery$count()}} and
188189
#' \href{#method-EsgfQuery-collect}{\code{EsgfQuery$collect()}}. It is a named
189190
#' list generated from the JSON response using [jsonlite::fromJSON()].
190191
#'
191192
#' - \href{#method-EsgfQuery-print}{\code{EsgfQuery$print()}}: Print a summary
192-
#' of the current `EsgfQuery` object including the host url, the built time of
193+
#' of the current `EsgfQuery` object including the host URL, the built time of
193194
#' facet cache and all query parameters.
194195
#'
195196
#' @author Hongyuan Jia
@@ -198,7 +199,7 @@ format.EsgfQueryParam <- function(x, encode = TRUE, space = FALSE, ...) {
198199
#'
199200
#' @param host The URL to the ESGF Search API service. This should be the URL of
200201
#' the ESGF search service excluding the final endpoint name. Usually
201-
#' this is `http://<hostname>/esg-search`. Default is to ses the
202+
#' this is `http://<hostname>/esg-search`. Default is set to the
202203
#' [LLNL (Lawrence Livermore National Laboratory) Index Node](http://esgf-node.llnl.gov),
203204
#' which is `"https://esgf-node.llnl.gov/esg-search"`.
204205
#'
@@ -1069,21 +1070,34 @@ EsgfQuery <- R6::R6Class("EsgfQuery",
10691070

10701071

10711072
#' @description
1072-
#' Get the actual query URL
1073+
#' Get the URL of actual query or wget script
1074+
#'
1075+
#' @param wget Whether to return the URL of the wget script that can be
1076+
#' used to download all files matching the given constraints.
1077+
#' Default: `FALSE`.
10731078
#'
10741079
#' @return A single string.
10751080
#'
10761081
#' @examples
10771082
#' \dontrun{
10781083
#' q$url()
1084+
#'
1085+
#' # get the wget script URL
1086+
#' q$url(wget = TRUE)
1087+
#'
1088+
#' # You can download the wget script using the URL directly. For
1089+
#' # example, the code below downloads the script and save it as
1090+
#' # 'wget.sh' in R's temporary folder:
1091+
#' download.file(q$url(TRUE), file.path(tempdir(), "wget.sh"), mode = "wb")
1092+
#'
10791093
#' }
1080-
url = function() {
1094+
url = function(wget = FALSE) {
1095+
assert_flag(wget)
10811096
params <- mget(ls(envir = private, pattern = "^param_"), envir = private)
10821097
names(params) <- gsub("^param_", "", names(params))
1083-
query_build(private$url_host, params)
1098+
query_build(private$url_host, params, type=if (wget) "wget"else"search")
10841099
},
10851100

1086-
10871101
#' @description
10881102
#' Send a query of facet counting and fetch the results
10891103
#'
@@ -1201,7 +1215,7 @@ EsgfQuery <- R6::R6Class("EsgfQuery",
12011215
#' Print a summary of the current `EsgfQuery` object
12021216
#'
12031217
#' `$print()` gives the summary of current `EsgfQuery` object including
1204-
#' the host url, the built time of facet cache and all query parameters.
1218+
#' the host URL, the built time of facet cache and all query parameters.
12051219
#'
12061220
#' @return The `EsgfQuery` object itself, invisibly.
12071221
#'
@@ -1389,11 +1403,14 @@ query_build <- function(host, params, type = "search") {
13891403
# skip empty parameter
13901404
params <- params[!vapply(params, is.null, logical(1L))]
13911405

1406+
if (type == "wget") {
1407+
params <- params[!names(params) %in% c("type", "format")]
1408+
}
1409+
13921410
if (!length(params)) return(NULL)
13931411

13941412
paste0(
1395-
sprintf("%s/search?", host),
1396-
if (type == "wget") "wget",
1413+
sprintf("%s/%s?", host, type),
13971414
paste(vapply(params, format.EsgfQueryParam, FUN.VALUE = ""), collapse = "&")
13981415
)
13991416
}

‎R/utils.R‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -72,7 +72,7 @@ rd_query_method_param <- function(method, type, negate, default, nullable = TRUE
7272
if (!missing(negate)) {
7373
sprintf(
7474
paste(
75-
"Note that you can put a preceding \\code{!} to negate the facet contraints.",
75+
"Note that you can put a preceding \\code{!} to negate the facet constraints.",
7676
"For example, \\code{$%s(!c(%s))} searches for all \\code{%s}s except for",
7777
"%s."
7878
),

‎README.Rmd‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -249,7 +249,7 @@ extent permitted by law.
249249

250250
If you encounter a clear bug or have questions about the usage, please file an
251251
issue with a minimal reproducible example on
252-
[GitHub](https://github.com/ideas-lab-nus/epwshiftr/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc)
252+
[GitHub](https://github.com/ideas-lab-nus/epwshiftr/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc)
253253
If you have a solution for an existing bug or an implementation for a missing
254254
feature, please send a pull request and let us review.
255255

0 commit comments

Comments
(0)

AltStyle によって変換されたページ (->オリジナル) /