Can't read data correctly through beeline when data is save by HiveContext

Previous Topic Next Topic
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

Can't read data correctly through beeline when data is save by HiveContext

This post has NOT been accepted by the mailing list yet.
This post was updated on .

        Here is my javacode;

      SparkConf sparkConf = Constance.getSparkConf();
        JavaSparkContext sc = new JavaSparkContext(sparkConf);
        SQLContext sql = new SQLContext(sc);
        HiveContext sqlContext = new HiveContext(;
        List<StructField> fields = new ArrayList<StructField>();
        fields.add(DataTypes.createStructField("id", DataTypes.IntegerType,
        List<Integer> data = Arrays.asList(1, 2, 3, 4, 5);
        JavaRDD<Integer> distData = sc.parallelize(data);
        StructType schema = DataTypes.createStructType(fields);
        JavaRDD<Row> rdd = Function<Integer, Row>() {
                        public Row call(Integer v1) throws Exception {
                                return RowFactory.create(v1);
        DataFrame peopleDataFrame = sqlContext.createDataFrame(rdd, schema);
        System.out.println("**************"+sqlContext.sql("select * from").toJavaRDD()

         bin/spark-submit --class com.bjdv.spark.demo.Copy_2_of_Fenlei  ~/sparkjob.jar

        I run this shell on my server to create a table named "" in Hive,the table have 5 records;
        And then I run "select * from" through beeline;

        It works well;
        But when I run the shell second or above;

        The javacode could read the data well,bug beeline still show only 5 records that insert by the first time;


        I'm sure that beeline and shell read the same hive-site.xml ,the metastore is saved with mysql;