Home
Projects
Resources
Alternatives
Blog
Sign In
Example Spark
Save
Spark, Spark Streaming and Spark SQL unit testing strategies
Overview
Reviews
Resources
Project README
Spark, Spark Streaming and Spark SQL unit testing strategies
Features
ClockWrapper
for efficient clock management in Spark Streaming jobs.
Base traits for testing
Spark
,
Spark Streaming
and
Spark SQL
to eliminate boilerplate code.
Sample applications to show how to make your code testable.
All tests can be run or debugged directly from IDE, or using SBT.
All test fixtures are prepared as in-memory data structures.
SBT is configured to avoid problems with multiple Spark contexts in the same JVM
SPARK-2243
.
SBT is configured to prepare project assembly for deployment on the cluster.
References
http://mkuthan.github.io/blog/2015/03/01/spark-unit-testing/
https://github.com/holdenk/spark-testing-base
Open Source Agenda is not affiliated with "Example Spark" Project. README Source:
mkuthan/example-spark
Stars
217
Open Issues
2
Last Commit
7 years ago
Repository
mkuthan/example-spark
Tags
Spark
Spark Streaming
Testing
Other
Open Source Agenda Badge
Submit Review
Review Your Favorite Project
Submit Resource
Articles, Courses, Videos
Submit Article
Submit a post to our blog
From the blog
Dec 11, 2022
How to Choose Which Programming Language to Learn First?
From the blog
Dec 11, 2022
How to Choose Which Programming Language to Learn First?
Home
Projects
Resources
Alternatives
Blog
Sign In
Sign In to OSA
I agree with
Terms of Service
and
Privacy Policy
Sign In with Github