Type: Package
Package: sparklyr
Title: R Interface to Apache Spark
Version: 1.9.4
Authors@R: 
    c(person(given = "Javier",
             family = "Luraschi",
             role = "aut",
             email = "jluraschi@gmail.com"),
      person(given = "Kevin",
             family = "Kuo",
             role = "aut",
             email = "kevin.kuo@rstudio.com",
             comment = c(ORCID = "0000-0001-7803-7901")),
      person(given = "Kevin",
             family = "Ushey",
             role = "aut",
             email = "kevin@rstudio.com"),
      person(given = "JJ",
             family = "Allaire",
             role = "aut",
             email = "jj@rstudio.com"),
      person(given = "Samuel",
             family = "Macedo",
             role = "ctb",
             email = "samuelmacedo@recife.ifpe.edu.br"),
      person(given = "Hossein",
             family = "Falaki",
             role = "aut",
             email = "hossein@databricks.com"),
      person(given = "Lu",
             family = "Wang",
             role = "aut",
             email = "lu.wang@databricks.com"),
      person(given = "Andy",
             family = "Zhang",
             role = "aut",
             email = "yue.zhang@databricks.com"),
      person(given = "Yitao",
             family = "Li",
             role = "aut",
             email = "yitaoli1990@gmail.com",
             comment = c(ORCID = "0000-0002-1261-905X")),
      person(given = "Jozef",
             family = "Hajnala",
             role = "ctb",
             email = "jozef.hajnala@gmail.com"),
      person(given = "Maciej",
             family = "Szymkiewicz",
             role = "ctb",
             email = "mszymkiewicz@gmail.com",
             comment = c(ORCID = "0000-0003-1469-9396")),
      person(given = "Wil",
             family = "Davis",
             role = "ctb",
             email = "william.davis@worthingtonindustries.com"),
      person(given = "Edgar",
             family = "Ruiz",
             role = c("aut", "cre"),
             email = "edgar@rstudio.com"),
      person(family = "RStudio",
             role = "cph"),
      person(family = "The Apache Software Foundation",
             role = c("aut", "cph")))
Maintainer: Edgar Ruiz <edgar@rstudio.com>
Description: R interface to Apache Spark, a fast and general
    engine for big data processing, see <https://spark.apache.org/>. This
    package supports connecting to local and remote Apache Spark clusters,
    provides a 'dplyr' compatible back-end, and provides an interface to
    Spark's built-in machine learning algorithms.
License: Apache License 2.0 | file LICENSE
URL: https://spark.posit.co/
BugReports: https://github.com/sparklyr/sparklyr/issues
Depends: R (>= 3.2)
Imports: config (>= 0.2), DBI (>= 1.0.0), dbplyr (>= 2.5.0), dplyr (>=
        1.0.9), generics, globals, glue, httr (>= 1.2.1), jsonlite (>=
        1.4), methods, openssl (>= 0.8), purrr, rlang (>= 0.1.4),
        rstudioapi (>= 0.10), tidyr (>= 1.2.0), tidyselect, uuid,
        vctrs, withr, xml2
Suggests: arrow (>= 0.17.0), broom, diffobj, foreach, ggplot2,
        iterators, janeaustenr, Lahman, mlbench, nnet, nycflights13,
        R6, r2d3, RCurl, reshape2, shiny (>= 1.0.1), parsnip, testthat,
        rprojroot
Encoding: UTF-8
RoxygenNote: 7.3.3
SystemRequirements: Spark: 2.x, or 3.x, or 4.x
Collate: 'spark_data_build_types.R' 'arrow_data.R' 'spark_invoke.R'
        'browse_url.R' 'spark_connection.R' 'avro_utils.R'
        'config_settings.R' 'config_spark.R' 'connection_instances.R'
        'connection_progress.R' 'connection_shinyapp.R'
        'spark_version.R' 'connection_spark.R' 'core_arrow.R'
        'core_config.R' 'core_connection.R' 'core_deserialize.R'
        'core_gateway.R' 'core_invoke.R' 'core_jobj.R'
        'core_serialize.R' 'core_utils.R' 'core_worker_config.R'
        'utils.R' 'sql_utils.R' 'data_copy.R' 'data_csv.R'
        'spark_schema_from_rdd.R' 'spark_apply_bundle.R'
        'spark_apply.R' 'tables_spark.R' 'tbl_spark.R' 'spark_sql.R'
        'spark_dataframe.R' 'dplyr_spark.R' 'sdf_interface.R'
        'data_interface.R' 'databricks_connection.R'
        'dbi_spark_connection.R' 'dbi_spark_result.R'
        'dbi_spark_table.R' 'do_spark.R' 'dplyr_do.R' 'dplyr_hof.R'
        'dplyr_join.R' 'dplyr_spark_data.R' 'dplyr_spark_table.R'
        'stratified_sample.R' 'sdf_sql.R' 'dplyr_sql.R'
        'dplyr_sql_translation.R' 'dplyr_verbs.R' 'imports.R'
        'install_spark.R' 'install_spark_versions.R'
        'install_spark_windows.R' 'install_tools.R' 'java.R'
        'jobs_api.R' 'kubernetes_config.R' 'shell_connection.R'
        'livy_connection.R' 'livy_install.R' 'livy_invoke.R'
        'livy_service.R' 'ml_clustering.R'
        'ml_classification_decision_tree_classifier.R'
        'ml_classification_gbt_classifier.R'
        'ml_classification_linear_svc.R'
        'ml_classification_logistic_regression.R'
        'ml_classification_multilayer_perceptron_classifier.R'
        'ml_classification_naive_bayes.R'
        'ml_classification_one_vs_rest.R'
        'ml_classification_random_forest_classifier.R'
        'ml_model_helpers.R' 'ml_clustering_bisecting_kmeans.R'
        'ml_clustering_gaussian_mixture.R' 'ml_clustering_kmeans.R'
        'ml_clustering_lda.R' 'ml_clustering_power_iteration.R'
        'ml_constructor_utils.R' 'ml_evaluate.R'
        'ml_evaluation_clustering.R' 'ml_evaluation_prediction.R'
        'ml_evaluator.R' 'ml_feature_binarizer.R'
        'ml_feature_bucketed_random_projection_lsh.R'
        'ml_feature_bucketizer.R' 'ml_feature_chisq_selector.R'
        'ml_feature_count_vectorizer.R' 'ml_feature_dct.R'
        'ml_feature_sql_transformer.R' 'ml_feature_dplyr_transformer.R'
        'ml_feature_elementwise_product.R'
        'ml_feature_feature_hasher.R' 'ml_feature_hashing_tf.R'
        'ml_feature_idf.R' 'ml_feature_imputer.R'
        'ml_feature_index_to_string.R' 'ml_feature_interaction.R'
        'ml_feature_lsh_utils.R' 'ml_feature_max_abs_scaler.R'
        'ml_feature_min_max_scaler.R' 'ml_feature_minhash_lsh.R'
        'ml_feature_ngram.R' 'ml_feature_normalizer.R'
        'ml_feature_one_hot_encoder.R'
        'ml_feature_one_hot_encoder_estimator.R' 'ml_feature_pca.R'
        'ml_feature_polynomial_expansion.R'
        'ml_feature_quantile_discretizer.R' 'ml_feature_r_formula.R'
        'ml_feature_regex_tokenizer.R' 'ml_feature_robust_scaler.R'
        'ml_feature_standard_scaler.R'
        'ml_feature_stop_words_remover.R' 'ml_feature_string_indexer.R'
        'ml_feature_string_indexer_model.R' 'ml_feature_tokenizer.R'
        'ml_feature_vector_assembler.R' 'ml_feature_vector_indexer.R'
        'ml_feature_vector_slicer.R' 'ml_feature_word2vec.R'
        'ml_fpm_fpgrowth.R' 'ml_fpm_prefixspan.R' 'ml_helpers.R'
        'ml_mapping_tables.R' 'ml_metrics.R' 'ml_model_als.R'
        'ml_model_bisecting_kmeans.R' 'ml_model_constructors.R'
        'ml_model_decision_tree.R' 'ml_model_gaussian_mixture.R'
        'ml_model_generalized_linear_regression.R'
        'ml_model_gradient_boosted_trees.R'
        'ml_model_isotonic_regression.R' 'ml_model_kmeans.R'
        'ml_model_lda.R' 'ml_model_linear_regression.R'
        'ml_model_linear_svc.R' 'ml_model_logistic_regression.R'
        'ml_model_naive_bayes.R' 'ml_model_one_vs_rest.R'
        'ml_model_random_forest.R' 'ml_model_utils.R'
        'ml_param_utils.R' 'ml_persistence.R' 'ml_pipeline.R'
        'ml_pipeline_utils.R' 'ml_print_utils.R'
        'ml_recommendation_als.R'
        'ml_regression_aft_survival_regression.R'
        'ml_regression_decision_tree_regressor.R'
        'ml_regression_gbt_regressor.R'
        'ml_regression_generalized_linear_regression.R'
        'ml_regression_isotonic_regression.R'
        'ml_regression_linear_regression.R'
        'ml_regression_random_forest_regressor.R' 'ml_stat.R'
        'ml_summary.R' 'ml_transformation_methods.R'
        'ml_transformer_and_estimator.R' 'ml_tuning.R'
        'ml_tuning_cross_validator.R'
        'ml_tuning_train_validation_split.R' 'ml_utils.R'
        'ml_validator_utils.R' 'mutation.R' 'na_actions.R'
        'new_model_multilayer_perceptron.R' 'params_validator.R'
        'precondition.R' 'project_template.R' 'qubole_connection.R'
        'reexports.R' 'sdf_dim.R' 'sdf_distinct.R' 'sdf_ml.R'
        'sdf_saveload.R' 'sdf_sequence.R' 'sdf_stat.R'
        'sdf_streaming.R' 'tidyr_utils.R' 'sdf_unnest_longer.R'
        'sdf_wrapper.R' 'sdf_unnest_wider.R' 'sdf_utils.R'
        'spark_compile.R' 'spark_context_config.R' 'spark_extensions.R'
        'spark_gateway.R' 'spark_gen_embedded_sources.R'
        'spark_globals.R' 'spark_hive.R' 'spark_home.R' 'spark_ide.R'
        'spark_submit.R' 'spark_update_embedded_sources.R'
        'spark_utils.R' 'spark_verify_embedded_sources.R'
        'stream_data.R' 'stream_job.R' 'stream_operations.R'
        'stream_shiny.R' 'stream_view.R' 'synapse_connection.R'
        'test_connection.R' 'tidiers_ml_aft_survival_regression.R'
        'tidiers_ml_als.R' 'tidiers_ml_isotonic_regression.R'
        'tidiers_ml_lda.R' 'tidiers_ml_linear_models.R'
        'tidiers_ml_logistic_regression.R'
        'tidiers_ml_multilayer_perceptron.R' 'tidiers_ml_naive_bayes.R'
        'tidiers_ml_svc_models.R' 'tidiers_ml_tree_models.R'
        'tidiers_ml_unsupervised_models.R' 'tidiers_pca.R'
        'tidiers_utils.R' 'tidyr_fill.R' 'tidyr_nest.R'
        'tidyr_pivot_utils.R' 'tidyr_pivot_longer.R'
        'tidyr_pivot_wider.R' 'tidyr_separate.R' 'tidyr_unite.R'
        'tidyr_unnest.R' 'tune-grid-spark.R' 'worker_apply.R'
        'worker_connect.R' 'worker_connection.R' 'worker_invoke.R'
        'worker_log.R' 'worker_main.R' 'yarn_cluster.R' 'yarn_config.R'
        'yarn_ui.R' 'zzz.R'
NeedsCompilation: no
Packaged: 2026-04-18 00:04:51 UTC; edgar
Author: Javier Luraschi [aut],
  Kevin Kuo [aut] (ORCID: <https://orcid.org/0000-0001-7803-7901>),
  Kevin Ushey [aut],
  JJ Allaire [aut],
  Samuel Macedo [ctb],
  Hossein Falaki [aut],
  Lu Wang [aut],
  Andy Zhang [aut],
  Yitao Li [aut] (ORCID: <https://orcid.org/0000-0002-1261-905X>),
  Jozef Hajnala [ctb],
  Maciej Szymkiewicz [ctb] (ORCID:
    <https://orcid.org/0000-0003-1469-9396>),
  Wil Davis [ctb],
  Edgar Ruiz [aut, cre],
  RStudio [cph],
  The Apache Software Foundation [aut, cph]
Repository: CRAN
Date/Publication: 2026-04-18 05:50:02 UTC
Built: R 4.6.0; ; 2026-04-29 19:27:35 UTC; windows
