I'm trying to use newAPIHadoopFile to implement custom InputFormat and RecordReader classes instead of using the default ones provided by Spark.
I can do the same in Java but am unable to find the correct way of achieving this using Mobius.
For reference, these are the classes and interfaces I want to override
java.lang.Object
org.apache.hadoop.mapreduce.InputFormat<K,V>
org.apache.hadoop.mapreduce.lib.input.FileInputFormat<K,V>
and Interface RecordReader<K,V>
https://hadoop.apache.org/docs/r2.4.1/api/org/apache/hadoop/mapreduce/RecordReader.html
https://hadoop.apache.org/docs/stable/api/index.htmlorg/apache/hadoop/mapreduce/lib/input/FileInputFormat.html
In Mobius, trying to use a custom InputFormat class throws a 'ClassNotFound' exception. I'm guessing it only excepts classes provided by Hadoop.
Does Mobius support custom partitioning ?
I'm trying to use newAPIHadoopFile to implement custom InputFormat and RecordReader classes instead of using the default ones provided by Spark.
I can do the same in Java but am unable to find the correct way of achieving this using Mobius.
For reference, these are the classes and interfaces I want to override
and
Interface RecordReader<K,V>https://hadoop.apache.org/docs/r2.4.1/api/org/apache/hadoop/mapreduce/RecordReader.html
https://hadoop.apache.org/docs/stable/api/index.htmlorg/apache/hadoop/mapreduce/lib/input/FileInputFormat.html
In Mobius, trying to use a custom InputFormat class throws a 'ClassNotFound' exception. I'm guessing it only excepts classes provided by Hadoop.
Does Mobius support custom partitioning ?