I am using spark 3.5.2 and I added the dependency google-auth-library-oauth2-http-1.33.1.jar
but this doesnt contain com/google/api/client/googleapis/auth/oauth2/GoogleCredential
so I found a jar that contains which is google-api-client-1.33.2.jar
but when I add this I get another NoClassDefError for com/google/api/client/auth/oauth2/Credential
so I am kind of stuck what jars is needed? my app simply writes from kafka to gcs bucket.
The question here is what are all the jars that are required to write to a GCS bucket?
25/03/19 20:56:01 INFO SparkContext: Successfully stopped SparkContext
Exception in thread "main" java.lang.NoClassDefFoundError: com/google/api/client/googleapis/auth/oauth2/GoogleCredential
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:467)
at .apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:2625)
at .apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2590)
at .apache.hadoop.conf.Configuration.getClass(Configuration.java:2686)
at .apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:3431)
at .apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3466)