iPlots eXtreme: next-generation interactive graphics design and implementation of modern interactive graphics

Interactive graphics provide a very important tool that facilitates the process of exploratory data and model analysis which is a crucial step in real-world applied statistics. Only a very limited set of software exists that provides truly interactive graphics for data analysis, partially because it is not easy to implement. Very often specialized software is created to offer graphics for a particular problem, but many fundamental plots are omitted since it is not considered new research. In this paper we discuss a general framework that allows to create interactive graphics software on a sound foundation that offers consistent user interface, fast prototyping of new plots and extensibility to support interactive models. In addition, we also discuss one implementation of the general framework: iPlots eXtreme—next-generation interactive graphics for analysis of large data in R. It provides most fundamental plot types and allows new interactive plots to be created. The implementation raises interactive graphics performance to an entirely new level. We will discuss briefly several methods that allowed us to achieve this goal and illustrate the use of advanced programmability features in conjunction with R.     

Introducing computational thinking through hands-on projects using R with applications to calculus,probability and data analysis

The goal of this paper is to promote computational thinking among mathematics, engineering, science and technology students, through hands-on computer experiments. These activities have the potential to empower students to learn, create and invent with technology, and they engage computational thinking through simulations, visualizations and data analysis. We present nine computer experiments and suggest a few more, with applications to calculus, probability and data analysis, which engage computational thinking through simulations, visualizations and data analysis. We are using the free (open-source) statistical programming language R. Our goal is to give a taste of what R offers rather than to present a comprehensive tutorial on the R language. In our experience, these kinds of interactive computer activities can be easily integrated into a smart classroom. Furthermore, these activities do tend to keep students motivated and actively engaged in the process of learning, problem solving and developing a better intuition for understanding complex mathematical concepts.     

The plumbing of interactive graphics

What is a pipeline, and why do we need one for interactive graphics? This conceptual paper attempts to answer these questions, building on previous work. A pipeline controls the transformation from data to graphical objects on our screens, and we argue that the pipeline must be present, in some form, in all graphics software. The pipeline is made explicit in descendants of DataViewer.     

Orca: A Visualization Toolkit for High-Dimensional Data

Abstract

This article describes constructing interactive and dynamic linked data views using the Java programming language. The data views are designed for data that have a multivariate component. The approach to displaying data comes from earlier research on building statistical graphics based on data pipelines, in which different aspects of data processing and graphical rendering are organized conceptually into segments of a pipeline. The software design takes advantage of the object-oriented nature of the Java language to open up the data pipeline, allowing developers to have greater control over their visualization applications. Importantly, new types of data views coded to adhere to a few simple design requirements can easily be integrated with existing pipe sections. This allows access to sophisticated linking and dynamic interaction across all (new and existing) view types. Pipe segments can be accessed from data analysis packages such as Omegahat or R, providing a tight coupling of visual and numerical methods.     

The S-Net System for Internet Packet Streams: Strategies for Stream Analysis and System Architecture

The traffic on an Internet link is a packet stream: packets of varying sizes arriving for transmission on the link. Each packet has an arrival time, and contained within the packet are headers that carry many critical variables. Packet traces, which consist of captured headers and measurements of the arrival times, convey substantial information about the Internet—security, usage, network performance, and the performance of engineering protocols. This article discusses strategies for the analysis of very large databases of packet traces, and the architecture of a software system that facilitates the use of these strategies. The system has a pipeline: (1) raw packet traces; (2) a database with objects tailored to ensuing analyses; and (3) an environment with tools for data analysis: statistical methods, model fitting, and visualization. The pipeline addresses the full set of tasks in the study of packet streams, from the initial processing of raw packet traces to the final output, often a visual display. S-Net—an extensible, open-source software implementation of this architecture—is based on the R implementation of the S language for graphics and data analysis, and has been developed on Linux.     

Role of Interaction in Enhancing the Epistemic Utility of 3D Mathematical Visualizations

Many epistemic activities, such as spatial reasoning, sense-making, problem solving, and learning, are information-based. In the context of epistemic activities involving mathematical information, learners often use interactive 3D mathematical visualizations (MVs). However, performing such activities is not always easy. Although it is generally accepted that making these visualizations interactive can improve their utility, it is still not clear what role interaction plays in such activities. Interacting with MVs can be viewed as performing low-level epistemic actions on them. In this paper, an epistemic action signifies an external action that modifies a given MV in a way that renders learners’ mental processing of the visualization easier, faster, and more reliable. Several, combined epistemic actions then, when performed together, support broader, higher-level epistemic activities. The purpose of this paper is to examine the role that interaction plays in supporting learners to perform epistemic activities, specifically spatial reasoning involving 3D MVs. In particular, this research investigates how the provision of multiple interactions affects the utility of 3D MVs and what the usage patterns of these interactions are. To this end, an empirical study requiring learners to perform spatial reasoning tasks with 3D lattice structures was conducted. The study compared one experimental group with two control groups. The experimental group worked with a visualization tool which provided participants with multiple ways of interacting with the 3D lattices. One control group worked with a second version of the visualization tool which only provided one interaction. Another control group worked with 3D physical models of the visualized lattices. The results of the study indicate that providing learners with multiple interactions can significantly affect and improve performance of spatial reasoning with 3D MVs. Among other findings and conclusions, this research suggests that one of the central roles of interaction is allowing learners to perform low-level epistemic actions on MVs in order to carry out higher-level cognitive and epistemic activities. The results of this study have implications for how other 3D mathematical visualization tools should be designed.     

Explorative and dynamic visualization of data in virtual reality

Summary  A software system has been developed for the study of dynamic glyph visualizations in the context of Visual Data Mining in Virtual Reality. The system uses parallel processing to calculate data visualizations in real-time, with real-time interaction and dynamic changes to the view. The system allows morphing between different visualizations, the use of dynamic features like “vibrations” and “rotations” of thousands of objects individually, and dynamic visualization, where the influence of any variable of a dataset with a “reasonable” distribution, can be shown as a dynamic development. It appears that these facilities for dynamic data visualization have a very promising potential, but their optimal use will depend on further developments in the context of their individual practical application.     

Lexical Scope and Statistical Computing

Abstract

Programming environments such as S and Lisp-Stat have languages for performing computations, data storage mechanisms, and a graphical interface. These languages provide an interactive interface to data analysis that is invaluable. To take full advantage of these programming environments, statisticians must understand the differences between them. Ihaka and Gentleman introduced R, a version of S which uses a different scoping regimen. In some ways this makes R behave more like Lisp-Stat. This article discusses the concept of scoping rules and shows how lexical scope can enhance the functionality of a language.     

Extending ggplot2 for Linked and Animated Web Graphics

Interactive web graphics are great for communication and knowledge sharing, but are difficult to leverage during the exploratory phase of a data science workflow. Even before the web, interactive graphics helped data analysts quickly gather insight from data, discover the unexpected, and develop better model diagnostics. Although web technologies make interactive graphics more accessible, they are not designed to fit inside an exploratory data analysis (EDA) workflow where rapid iteration between data manipulation, modeling, and visualization must occur. To better facilitate exploratory web graphics that are easily distributed, we need better interfaces between statistical computing environments (e.g., the R language) and client-side web technologies. We propose the R package animint for rapid creation of linked and animated web graphics through a simple extension of ggplot2’s implementation of the Grammar of Graphics. The extension allows one to write ggplot2 code and produce a standalone web page with multiple linked views. Supplementary material for this article is available online.     

The Plot-Data Interface in Statistical Graphics

Abstract

Statistical software systems include modules for manipulating data sets, model fitting, and graphics. Because plots display data, and models are fit to data, both the model-fitting and graphics modules depend on the data. Today's statistical environments allow the analyst to choose or even build a suitable data structure for storing the data and to implement new kinds of plots. The multiplicity problem caused by many plot varieties and many data representations is avoided by constructing a plot-data interface. The interface is a convention by which plots communicate with data sets, allowing plots to be independent of the actual data representation. This article describes the components of such a plot-data interface. The same strategy may be used to deal with the dependence of model-fitting procedures on data.     

The NNTMM code: Mathematical modeling, optimization, and data analysis through neural networks

The concept, functional capabilities, graphic interface, and application technology for the NNTMM code (Neural Network Tool for Mathematical Modeling), designed for analyzing large amounts of experimental and computational data in an interactive mode, are described. The software can be used to solve a wide range of applied problems where automated extraction of meaningful information from randomly presented data is required, along with data classification, clear representation of established relationships and dependences, analysis of parameter importance, and prediction. Examples of using the NNTMM code to analyze data in controlled nuclear fusion problems are presented.     

A modest proposal: an approach to making the internal R system extensible

The R computing environment has become an important part of the statistical community and fostered the development of over a thousand add-on packages, many representing state-of-the-art research in statistical methodology. Although it is relatively easy to develop functionality on top of the system, it is very difficult for developers to directly extend the core system itself—the language, the interpreter and the internal data structures. Yet the ability to easily introduce new core, first-class data structures into the system that are customized and efficient is becoming essential in this era of large, complex data sets and innovative algorithms and data structures. While the community that might use such a facility to introduce new data types may be small, it is potentially very talented and important, and may lead to significant innovations that allow us to continue to leverage R for the next 5 years or more in rich new ways. I describe some of the difficulties that people encounter in extending the system and suggest that an object-oriented architecture for the internal implementation of R (or any system) would make such low-level internals extensible by package developers and not just the core development team. This would promote potentially rich experimentation that would allow us and others to approach new styles of computation in R, while simultaneously maintaining the existing important community which provides so much value-added to the R environment. Specifically, transforming the R implementation from a representation-specific architecture to a C++ abstract/virtual interface-based architecture may be the least disruptive approach to the continued evolution of R, and would bring many advantages and some technical challenges. Such an approach involves many technical details and potential degradations in performance. Due to the length of the this paper, I do not explore these issues in great detail but introduce the basic concepts. I do, however, refer to some technical aspects that are best understood with some knowledge of the implementation of R at the level of using the .Call () interface in R.     

复变函数的可视化问题

首先简单介绍了复变函数基础理论的内容可视化和其基于数学软件的直观演示资源建设情况,然后通过幅角函数可视化问题介绍利用数学软件开发应用于数学教学实践的演示素材的方法,解决了Mathematica中的标准幅角函数不能直接表示复数幅角变化量的问题.     

An Interactive Dynamic Inventory-Production Control System

The development and evaluation of an interactive model for controlling inventories and scheduling production in a high technology batch production environment are discussed. The model allows the use of expert knowledge and company policy for matters such as customer service and safety stocks, in an interactive way. Production of a large number of products has to be scheduled for a number of time periods and a modified Wagner-Whitin algorithm is used for developing good schedules. A versatile data and information management system is a part of the developed software which has been implemented using a standard microcomputer. An evaluation suggests that the system is a practical one for use by busy managers. Further, it results in substantial cost savings and increased customer service.     

Determining a subsidy rate for Taiwan's recycling glass industry: an application of bi-level programming

This study attempts to optimize the operations of the Recycling Fund Management Board (RFMB), founded by the Environmental Protection Administration of the R.O.C. Government (on Taiwan), through the decision of a subsidy rate for the domestic glass recycling industry. The hierarchical and interactive nature between the two parties is modelled by bi-level programming, where the RFMB plays the upper-level decision unit while the recycling industry is the lower-level counterpart. In order to solve the problem by optimization software, the bi-level formulation is transformed to a single-level problem via Karush-Kuhn-Tucker optimality conditions and is further transformed to a 0?1 mixed integer programming problem by variable substitution. The problem is solved with real-world data, and the obtained solutions are analysed and compared with the RFMB’s current operations. The results suggest that the proposed approach can improve the operations of the RFMB.     

Enabling Interactivity on Displays of Multivariate Time Series and Longitudinal Data

Temporal data are information measured in the context of time. This contextual structure provides components that need to be explored to understand the data and that can form the basis of interactions applied to the plots. In multivariate time series, we expect to see temporal dependence, long term and seasonal trends, and cross-correlations. In longitudinal data, we also expect within and between subject dependence. Time series and longitudinal data, although analyzed differently, are often plotted using similar displays. We provide a taxonomy of interactions on plots that can enable exploring temporal components of these data types, and describe how to build these interactions using data transformations. Because temporal data are often accompanied other types of data we also describe how to link the temporal plots with other displays of data. The ideas are conceptualized into a data pipeline for temporal data and implemented into the R package cranvas. This package provides many different types of interactive graphics that can be used together to explore data or diagnose a model fit.     

Short-term Operation Planning in Electric Power Systems

A practical application of operational research methods is shown here for the short-term operation planning in an electric power system. Two interactive application programs are presented: a short-term load forecasting program and a unit commitment optimisation program. For the load prediction within the load forecasting program a four-component weather dependent load model is used, whereby for the weather-dependent load component an econometric model, and for the time-dependent load component a seasonal multiplicative ARIMA-model is applied. The unit commitment is optimized using dynamic programming. With this method the start-up and shut-down costs can easily be taken into consideration. Special features are necessary to allow the optimal scheduling of a large number of generators to minimize storage requirements and execution time of the program. For every cost-optimal unit commitment determined the reliability of the energy supply is then calculated.The success of the application software depends to a large extent on the man-machine interface. Special attention has to be paid to the man-machine dialogue. The dialogue has to be structured in such a way that the user has the facility to control the programs as easily as possible.All input data has to be checked and in case of a false input an error message has to be displayed and a new input requested. The data-base has to be protected against wrong actions.     

A software package for the simulation of second-harmonic generation in the three-dimensional case and visualization of the results in a windows environment

We describe a software package for visualizing the simulation results of second-harmonic generation in a medium with combined (quadratic-cubic) nonlinearity in the (z, r, t) coordinates. The package includes a wide range of graphical tools that create avi files, display intensity distributions of interacting waves, generate three-dimensional graphs with varying degree of detail, and plot various characteristics of the optical radiation. The package provides a user-friendly interface for analyzing a wide range of solutions and estimating the efficiency of energy transformation in the parameter plane that characterizes wave interaction. To this end the package represents the analytical solution of the problem in the long-pulse and plane-wave approximation allowing for quadratic and cubic nonlinearity of the medium. The analytical solution can be used also for testing simulation results. The software package is equipped with an interface for computing dimensionless parameters for the corresponding system of dimensionless Schrödinger equations from physical parameters specified in one of the systems of physical units. Options for DOS and Unix format conversions are provided, as the original computer program solving the nonlinear Schrödinger equations is designed for either Windows or Unix environments. The package has been applied to analyze various second-harmonic generation modes in a medium with combined (quadratic-cubic) nonlinearity.     

Natural Selection: Interactive Subset Creation

Abstract

Visualization is a critical technology for understanding complex, data-rich systems. Effective visualizations make important features of the data immediately recognizable and enable the user to discover interesting and useful results by highlighting patterns. A key element of such systems is the ability to interact with displays of data by selecting a subset for further investigation. This operation is needed for use in linked views systems and in drill-down analysis. It is a common manipulation in many other systems and is as ubiquitous as selecting icons in a desktop graphical user interface (GUI). It is therefore surprising to note that little research has been done on how selection can be implemented. This article addresses this omission, presenting a taxonomy for selection mechanisms and discussing the interactions between branches of the taxonomy.