作者计算机毕业设计小途个人简介曾长期从事计算机专业培训教学本人也热爱上课教学语言擅长Java、微信小程序、Python、Golang、安卓Android等开发项目包括大数据、深度学习、网站、小程序、安卓、算法。平常会做一些项目定制化开发、代码讲解、答辩教学、文档编写、也懂一些降重方面的技巧。平常喜欢分享一些自己开发中遇到的问题的解决办法也喜欢交流技术大家有技术代码这一块的问题可以问我想说的话感谢大家的关注与支持网站实战项目安卓/小程序实战项目大数据实战项目深度学习实战项目目录豆瓣图书数据分析与可视化系统介绍豆瓣图书数据分析与可视化系统演示视频豆瓣图书数据分析与可视化系统演示图片豆瓣图书数据分析与可视化系统代码展示豆瓣图书数据分析与可视化系统文档展示豆瓣图书数据分析与可视化系统介绍《基于大数据的豆瓣图书数据分析与可视化》是一个面向豆瓣图书数据的毕业设计系统主要用Hadoop和Spark做大数据处理底层通过HDFS保存原始图书数据用Spark SQL完成清洗、分组、统计和聚合再结合Pandas、NumPy做指标计算分析结果落到MySQL中后端可选PythonDjango或JavaSpring Boot实现接口前端用Vue、ElementUI、ECharts等展示图表。系统功能包括系统首页、大屏可视化、豆瓣图书分析、类型分布、创作状态、作者产出、风格偏好、作品热度、时间趋势、群体画像、用户管理、小说信息、个人信息和修改密码等。它把图书类型、作者产出、作品热度、评分变化、出版时间趋势和读者群体特征这些内容整理成图表和指标帮助使用者从大数据角度了解豆瓣图书的整体情况。大屏可视化可以集中展示核心统计结果普通分析页面则按主题查看类型分布、风格偏好、作品热度和时间趋势用户模块负责登录后的个人信息维护和密码修改。整体上这个系统把数据采集后的离线分析、指标统计、结果存储和可视化展示串成一条完整流程。豆瓣图书数据分析与可视化系统演示视频项目演示视频豆瓣图书数据分析与可视化系统演示图片豆瓣图书数据分析与可视化系统代码展示sparkSparkSession.builder.appName(DoubanBookBigData).master(local[*]).config(spark.sql.shuffle.partitions,4).getOrCreate()defanalyze_type_distribution():book_dfspark.read.option(header,true).option(inferSchema,true).csv(hdfs:///douban/book/books.csv)book_dfbook_df.filter(type is not null and type )book_df.cache()book_df.createOrReplaceTempView(douban_book)type_dfspark.sql(SELECT type, COUNT(*) AS book_count, ROUND(AVG(rating), 2) AS avg_rating FROM douban_book GROUP BY type ORDER BY book_count DESC)type_dftype_df.filter(book_count 0)pdftype_df.toPandas()pdf[percent](pdf[book_count]/pdf[book_count].sum()*100).round(2)pdf[rank_no]pdf[book_count].rank(methoddense,ascendingFalse).astype(int)pdf[avg_rating]pdf[avg_rating].fillna(0).round(2)recordspdf.to_dict(orientrecords)cursormysql_conn.cursor()cursor.execute(DELETE FROM book_type_stat)cursor.executemany(REPLACE INTO book_type_stat(type_name, book_count, avg_rating, percent, rank_no) VALUES(%s, %s, %s, %s, %s),[(r[type],int(r[book_count]),float(r[avg_rating]),float(r[percent]),int(r[rank_no]))forrinrecords])mysql_conn.commit()cursor.close()book_df.unpersist()return{code:200,data:records,total:len(records)}defanalyze_book_hot():hot_dfspark.read.option(header,true).option(inferSchema,true).csv(hdfs:///douban/book/books.csv)hot_dfhot_df.filter(rating is not null and rating_count 0 and comment_count is not null)hot_df.createOrReplaceTempView(book_hot)rank_dfspark.sql(SELECT title, author, rating, rating_count, comment_count, (rating * 0.6 LOG(rating_count 1) * 0.25 LOG(comment_count 1) * 0.15) AS hot_score FROM book_hot ORDER BY hot_score DESC LIMIT 50)rank_dfrank_df.filter(hot_score is not null)pdfrank_df.toPandas()pdf[rating]pdf[rating].round(2)pdf[hot_score]pdf[hot_score].round(2)pdf[hot_level]numpy.where(pdf[hot_score]pdf[hot_score].quantile(0.8),高热,numpy.where(pdf[hot_score]pdf[hot_score].quantile(0.5),中热,普通))pdf[hot_rank]pdf[hot_score].rank(methodmin,ascendingFalse).astype(int)recordspdf.to_dict(orientrecords)cursormysql_conn.cursor()cursor.execute(DELETE FROM book_hot_rank)cursor.executemany(REPLACE INTO book_hot_rank(title, author, rating, rating_count, comment_count, hot_score, hot_level, hot_rank) VALUES(%s, %s, %s, %s, %s, %s, %s, %s),[(r[title],r[author],float(r[rating]),int(r[rating_count]),int(r[comment_count]),float(r[hot_score]),r[hot_level],int(r[hot_rank]))forrinrecords])mysql_conn.commit()cursor.close()return{code:200,data:records,total:len(records)}defanalyze_time_trend():trend_dfspark.read.option(header,true).option(inferSchema,true).csv(hdfs:///douban/book/books.csv)trend_dftrend_df.filter(publish_year is not null and rating_count is not null)trend_df.createOrReplaceTempView(book_trend)year_dfspark.sql(SELECT publish_year, COUNT(*) AS publish_count, ROUND(AVG(rating), 2) AS avg_rating, SUM(rating_count) AS total_rating_count FROM book_trend WHERE publish_year BETWEEN 2000 AND 2026 GROUP BY publish_year ORDER BY publish_year)year_dfyear_df.filter(publish_count 0)pdfyear_df.toPandas()pdfpdf.sort_values(publish_year)pdf[moving_avg]pdf[publish_count].rolling(window3,min_periods1).mean().round(2)pdf[diff_value]pdf[moving_avg].diff().fillna(0).round(2)pdf[trend_flag]numpy.where(pdf[diff_value]0,上升,numpy.where(pdf[diff_value]0,下降,平稳))pdf[avg_rating]pdf[avg_rating].fillna(0).round(2)recordspdf.to_dict(orientrecords)cursormysql_conn.cursor()cursor.execute(DELETE FROM book_time_trend)cursor.executemany(REPLACE INTO book_time_trend(publish_year, publish_count, avg_rating, total_rating_count, moving_avg, trend_flag) VALUES(%s, %s, %s, %s, %s, %s),[(int(r[publish_year]),int(r[publish_count]),float(r[avg_rating]),int(r[total_rating_count]),float(r[moving_avg]),r[trend_flag])forrinrecords])mysql_conn.commit()cursor.close()return{code:200,data:records,total:len(records)}豆瓣图书数据分析与可视化系统文档展示作者计算机毕业设计小途个人简介曾长期从事计算机专业培训教学本人也热爱上课教学语言擅长Java、微信小程序、Python、Golang、安卓Android等开发项目包括大数据、深度学习、网站、小程序、安卓、算法。平常会做一些项目定制化开发、代码讲解、答辩教学、文档编写、也懂一些降重方面的技巧。平常喜欢分享一些自己开发中遇到的问题的解决办法也喜欢交流技术大家有技术代码这一块的问题可以问我想说的话感谢大家的关注与支持网站实战项目安卓/小程序实战项目大数据实战项目深度学习实战项目