The Art and Science of Analyzing Software Data

The Art and Science of Analyzing Software Data provides valuable information on analysis techniques often used to derive insight from software data. This book shares best practices in the field generated by leading data scientists, collected from their experience training software engineering students and practitioners to master data science.

The book covers topics such as the analysis of security data, code reviews, app stores, log files, and user telemetry, among others. It covers a wide variety of techniques such as co-change analysis, text analysis, topic analysis, and concept analysis, as well as advanced topics such as release planning and generation of source code comments. It includes stories from the trenches from expert data scientists illustrating how to apply data analysis in industry and open source, present results to stakeholders, and drive decisions.

Presents best practices, hints, and tips to analyze data and apply tools in data science projectsPresents research methods and case studies that have emerged over the past few years to further understanding of software dataShares stories from the trenches of successful data science initiatives in industry

1. Past, Present, and Future of Analyzing Software Data

Part 1 TUTORIAL-TECHNIQUES

2. Mining Patterns and Violations Using Concept Analysis
3. Analyzing Text in Software Projects
4. Synthesizing Knowledge from Software Development Artifacts
5. A Practical Guide to Analyzing IDE Usage Data
6. Latent Dirichlet Allocation: Extracting Topics from Software Engineering Data
7. Tools and Techniques for Analyzing Product and Process Data

PART 2 DATA/PROBLEM FOCUSSED

8. Analyzing Security Data
9. A Mixed Methods Approach to Mining Code Review Data: Examples and a Study of Multicommit Reviews and Pull Requests
10. Mining Android Apps for Anomalies
11. Change Coupling Between Software Artifacts: Learning from Past Changes

PART 3 STORIES FROM THE TRENCHES

12. Applying Software Data Analysis in Industry Contexts: When Research Meets Reality
13. Using Data to Make Decisions in Software Engineering:
14. Providing a Method to our Madness
15. Community Data for OSS Adoption Risk Management
16. Assessing the State of Software in a Large Enterprise: A 12-Year Retrospective
17. Lessons Learned from Software Analytics in Practice

PART 4 ADVANCED TOPICS

18. Code Comment Analysis for Improving Software Quality
19. Mining Software Logs for Goal-Driven Root Cause Analysis
20. Analytical Product Release Planning

PART 5 DATA ANALYSIS AT SCALE (BIG DATA)

21. Boa: An Enabling Language and Infrastructure for Ultra-Large-Scale MSR Studies
22. Scalable Parallelization of Specification Mining Using Distributed Computing