A latent model for collaborative filtering

Recommender systems based on collaborative filtering have received a great deal of interest over the last two decades. In particular, recently proposed methods based on dimensionality reduction techniques and using a symmetrical representation of users and items have shown promising results. Following this line of research, we propose a probabilistic collaborative filtering model that explicitly represents all items and users simultaneously in the model. Experimental results show that the proposed system obtains significantly better results than other collaborative filtering systems (evaluated on the MovieLens data set). Furthermore, the explicit representation of all users and items allows the model to e.g. make group-based recommendations balancing the preferences of the individual users.     

融入用户社交关系与信任关系的应用推荐方法研究

随着近年来互联网技术的快速发展,应用获取平台都面临着信息过载的问题.面对大量应用,解决用户不能快速准确地找到满足其偏好的应用的问题迫在眉睫.已有的如Cosine、Pearson等协同过滤方法普遍存在稀疏性、冷启动和可扩展性等问题,从而对推荐结果产生影响.文章在考虑用户社交关系、偏好及信任关系的基础上,提出了融合用户社交...     

A problem of data mining in E-commerce

Data mining is generally defined as the science of nontrivial extraction of implicit, previously unknown, and potentially useful information from datasets. There are many websites on the Internet that provide extensive information about products and allow users post comments on various products and rate the product on a scale of 1 to 5. During the past decade, the need for intelligent algorithms for calculating and organizing extremely large sets of data has grown exponentially. In this article we investigate the extent to which a product’s average user rating can be predicted, using a manageable subset of a data set. For this we use a linearization-algorithm based prediction model and sketch how an inverse problem can be formulated to yield a smooth local volatility function of user ratings. The MAPLE programs that implement the proposed algorithm show that the method is reasonably accurate for the reconstruction of volatility of user ratings, which is useful both in accurate user predictions as well as computing sensitivity.     

User acceptance of multi-criteria decision support systems: The impact of preference elicitation techniques

Previous research indicates that decision makers are often reluctant to use potentially beneficial multi-criteria decision support systems (MCDSS). Prior research has not examined the specific impact of preference elicitation techniques on user acceptance of MCDSS. The present research begins to fill this gap by examining the effect on users’ MCDSS evaluations of two commonly used preference elicitation techniques, absolute measurement and pairwise comparisons, while holding constant all other aspects of the MCDSS and decision making task. Experimental results (N = 153) indicate that users consider MCDSS with pairwise comparisons to be higher in decisional conflict, more effortful, less accurate, and overall less desirable to use than MCDSS with absolute measurements. Thus, any potential normative superiority of a preference elicitation technique must be balanced against its potentially adverse effects on user acceptance of the MCDSS within which it is employed. We present a research agenda for exploring the tradeoffs between objective validity and user acceptance in the design of decision analysis tools.     

Differentially Private Significance Tests for Regression Coefficients

Many data producers seek to provide users access to confidential data without unduly compromising data subjects’ privacy and confidentiality. One general strategy is to require users to do analyses without seeing the confidential data; for example, analysts only get access to synthetic data or query systems that provide disclosure-protected outputs of statistical models. With synthetic data or redacted outputs, the analyst never really knows how much to trust the resulting findings. In particular, if the user did the same analysis on the confidential data, would regression coefficients of interest be statistically significant or not? We present algorithms for assessing this question that satisfy differential privacy. We describe conditions under which the algorithms should give accurate answers about statistical significance. We illustrate the properties of the proposed methods using artificial and genuine data. Supplementary materials for this article are available online.     

Natural Selection: Interactive Subset Creation

Abstract

Visualization is a critical technology for understanding complex, data-rich systems. Effective visualizations make important features of the data immediately recognizable and enable the user to discover interesting and useful results by highlighting patterns. A key element of such systems is the ability to interact with displays of data by selecting a subset for further investigation. This operation is needed for use in linked views systems and in drill-down analysis. It is a common manipulation in many other systems and is as ubiquitous as selecting icons in a desktop graphical user interface (GUI). It is therefore surprising to note that little research has been done on how selection can be implemented. This article addresses this omission, presenting a taxonomy for selection mechanisms and discussing the interactions between branches of the taxonomy.     

Reputation,inequality and meeting techniques: visualising user hierarchy to support collaboration

Equality within groups is ordinarily taken for granted when technology for e-democracy is conceived and developed. However, inequality in online communication is just as common as in other social contexts. Therefore, we have developed a groupware with the express purpose of illuminating imbalance of power. Inequalities are measured and made visible to users of the system, and they change dynamically as actions are taken by users. The system is based on democratic meeting techniques and is reminiscent of a strategy game based on social media. Each participant’s score within the game is dynamically calculated and reflects that user’s activity, others’ reactions to that activity and reactions to others’ activities. The calculations and weighing mechanisms are open to inspection and change by the users, and hierarchical roles reflecting game levels may be attached to system rights belonging to individual users and user groups. The prototype we present stems from the question of how to conceive of groupware based on diversity and is the result of combining social theory with algorithms for modelling and visualising user hierarchy and status. Empirical user tests suggest improvements to the prototype’s interface, which will be implemented and further evaluated by embedding the algorithms in a system for e-participation.     

Forecaster—an Exercise in User Friendliness

This paper considers the factors that help to make a package ‘user friendly’ and hence more acceptable to users in management. The paper is based on experience with a forecasting package ‘Forecaster’.     

Policy-based inconsistency management in relational databases

Though inconsistency management in databases and AI has been studied extensively for years, it does not allow the user to specify how he wants to resolve inconsistencies. In real-world applications, users may want to manage or resolve inconsistencies based not only on the data, but their own knowledge of the risks involved in decision making based on faulty data. Each user should be empowered to use reasonable policies to deal with his data and his mission needs. In this paper, we start by providing an axiomatic definition of inconsistency management policies (IMPs) that puts this power in the hands of users. Any function satisfying these axioms is an IMP. We then define three broad families of IMPs, and derive several results that show (i) how these policies relate to postulates for the revision of belief bases and to recent research in the area of consistent query answering, and (ii) how they interact with standard relational algebra operators. Finally, we present several approaches to efficiently implement an IMP-based framework.     

Intelligent Support for Exploratory Data Analysis

Abstract

Exploratory data analysis (EDA) is as much a matter of strategy as of selecting specific statistical operations. We have developed a knowledge-based planning system, called AIDE, to help users with EDA. AIDE strikes a balance between conventional statistical packages, which need guidance for every step in the exploration, and autonomous systems, which leave the user entirely out of the decision-making process. AIDE's processing is based on artificial intelligence planning techniques, which give us a useful means of representing some types of statistical strategy. In this article we describe the design of AIDE and its behavior in exploring a small, complex data set.     

A radial basis function method for solving PDE-constrained optimization problems

In this article, we apply the theory of meshfree methods to the problem of PDE-constrained optimization. We derive new collocation-type methods to solve the distributed control problem with Dirichlet boundary conditions and also discuss the Neumann boundary control problem, both involving Poisson’s equation. We prove results concerning invertibility of the matrix systems we generate, and discuss a modification to guarantee invertibility. We implement these methods using Matlab, and produce numerical results to demonstrate the methods’ capability. We also comment on the methods’ effectiveness in comparison to the widely-used finite element formulation of the problem, and make some recommendations as to how this work may be extended.     

The Omegahat Environment: New Possibilities for Statistical Computing

Abstract

Statistical environments such as S, R, XLisp-Stat, and others have had a dramatic effect on the way we, statistics practitioners, think about data and statistical methodology. However, the possibilities and challenges introduced by recent technological developments and the general ways we use computing conflict with the computational model of these systems. This article explores some of these challenges and identifies the need to support easy integration of functionality from other domains, and to export statistical methodology to other audiences and applications, both statically and dynamically. Existing systems can be improved in these domains with some already implemented extensions (see Section 5). However, the development of a new environment and computational model that exploits modern tools designed to handle many general aspects of these challenges appears more promising as a long-term approach. We present the architecture for such a new model named Omegahat. It lends itself to entirely new statistical computing paradigms. It is highly extensible at both the user and programmer level, and also encourages the development of new environments for different user groups. The Omegahat interactive language offers a continuity between the different programming tasks and levels via optional type checking and seamless access between the interpreted user language and the implementation language, Java. Parallel and distributed computing, network and database access, interactive graphics, and many other aspects of statistical computing are directly accessible to the user as a consequence of this seamless access. We describe the benefits of using Java as the implementation language for the environment and several innovative features of the user-level language which promise to assist development of software that can be used in many contexts. We also outline how this architecture can be integrated with existing environments such as R and S.

The ideas are drawn from work within the Omega Project for Statistical Computing. The project provides open-source software for researching and developing next generation statistical computing tools.     

Utilizing the IP Multimedia Subsystem to create an extensible service-oriented architecture

Smart Spaces provide very promising means of creating context-aware environments. Unfortunately, a lack of information about users within Smart Spaces limits their usefulness. We propose a novel solution that involves integrating Smart Spaces with social networks through the IP Multimedia Subsystem. to create truly context-aware and adaptive spaces. By utilizing the wealth of user information present within social networks, smarter and more adaptive spaces can be created. We therefore propose the design and implementation of “SocioSpace” a Smart Spaces framework that utilizes the social context. We design and implement all components of SocioSpace, including the central server, the location management system, social network interfacing components, service delivery server and user agents. We then run various scenarios to test the reliability of the system. The results show the effectiveness of our framework in successfully creating Smart Spaces that can truly utilize social networks to deliver adaptive services that enhance the users’ experiences and make the environment more beneficial to them.     

Model Based Comparison of Discounted Cumulative Gain and Average Precision

In this paper, we propose to explain Discounted Cumulative Gain (DCG) as the expectation of the total utility collected by a user given a generative probabilistic model on how users browse the result page ranking list of a search engine. We contrast this with a generalization of Average Precision, pAP, that has been defined in Dupret and Piwowarski (2010) [13]. In both cases, user decision models coupled with Web search logs allow to estimate some parameters that are usually left to the designer of a metric. In this paper, we compare the user models for DCG and pAP at the interpretation and experimental level.DCG and AP are metrics computed before a ranking function is exposed to users and as such, their role is to predict the function performance. In counterpart to prognostic metric, a diagnostic metric is computed after observing the user interactions with the result list. A commonly used diagnostic metric is the clickthrough rate at position 1, for example. In this work we show that the same user model developed for DCG can be used to derive a diagnostic version of this metric. The same hold for pAP and any metric with a proper user model.We show that not only does this diagnostic view provide new information, it also allows to define a new criterion for assessing a metric. In previous works based on user decision modeling, the performance of different metrics were compared indirectly in terms of the ability of the associated user model to predict future user actions. Here we propose a new and more direct criterion based on the ability of the prognostic version of the metric to predict the diagnostic performance.     

Analysing and exploiting the Mantin biases in RC4

We explore the use of the Mantin biases (Mantin, Eurocrypt 2005) to recover plaintexts from RC4-encrypted traffic. We provide a more fine-grained analysis of these biases than in Mantin’s original work. We show that, in fact, the original analysis was incorrect in certain cases: the Mantin biases are sometimes non-existent, and sometimes stronger than originally predicted. We then show how to use these biases in a plaintext recovery attack. Our attack targets two unknown bytes of plaintext that are located close to sequences of known plaintext bytes, a situation that arises in practice when RC4 is used in, for example, TLS. We provide a statistical framework that enables us to make predictions about the performance of this attack and its variants. We then extend the attack using standard dynamic programming techniques to tackle the problem of recovering longer plaintexts, a setting of practical interest in recovering HTTP session cookies and user passwords that are protected by RC4 in TLS. We perform experiments showing that we can successfully recover 16-byte plaintexts with 80% success rate using \(2^{31}\) ciphertexts, an improvement over previous attacks.     

Book Reviews

The making of pictures and the use of mathematics are often considered as activities carried out by two different classes of people.

It may be true that the artist can get on without mathematics, but the converse is far less true.

The operation which an artist terms ‘drawing’, might be described by a mathematician as ‘the mapping of a three‐dimensional network into a two‐dimensional one’.

This article attempts to show how the mathematically minded student can use his mathematics to manipulate pictures. In doing so it introduces him to the tasks which a computer must perform in picture manipulation.

The article is in two parts:

Part A, discusses the use of three‐dimensional sketching and the role it plays in the preparation of ‘orthographic’ working drawings.

It describes how a designer transfers his thoughts about spacial objects to paper, thus assisting himself to refine them and enabling others to perceive them.

A case is made for encouraging perspective sketching in the teaching of engineering drawing.

Part B describes a technique for plotting perspective sketches by numerical methods, which may be useful in motivating numerically inclined students towards involvement with perspective sketching.     

Languages for Statistics and Data Analysis

Abstract

Languages for data analysis and statistics must be able to cover the entire spectrum from improvisation and fast prototyping to the implementation of streamlined, specialized systems for routine analyses. Such languages must not only be interactive but also programmable, and the distinctions between language, operating system, and user interface get blurred. The issues are discussed in the context of natural and computer languages, and of the different types of user interfaces (menu, command language, batch). It is argued that while such languages must have a completely general computing language kernel, they will contain surprisingly few items specific to data analysis—the latter items more properly belong to the “literature” (i.e., the programs) written in the language.     

Establishing wireless conference calls under delay constraints

A prevailing feature of mobile telephony systems is that the cell where a mobile user is located may be unknown. Therefore, when the system is to establish a call between users, it may need to search, or page, all the cells that it suspects the users are located in, to find the cells where the users currently reside. The search consumes expensive wireless links and so it is desirable to develop search techniques that page as few cells as possible.We consider cellular systems with c cells and m mobile users roaming among the cells. The location of the users is uncertain as given by m probability distribution vectors. Whenever the system needs to find specific users, it conducts a search operation lasting some number of rounds (the delay constraint). In each round, the system may check an arbitrary subset of cells to see which users are located there. In this setting the problem of finding one user with minimum expected number of cells searched is known to be solved optimally in polynomial time.In this paper we address the problem of finding several users with the same optimization goal. This task is motivated by the problem of establishing a conference call between mobile users. We first show that the problem is NP-hard. Then we prove that a natural heuristic is an e/(e−1)-approximation solution.     

Users,Programmers, and Statistical Software

Abstract

Statistical software provides essential support for statisticians and others who are analyzing data or doing research on new statistical techniques. Those supported typically regard themselves as “users” of the software, but as soon as they need to express their own ideas computationally, they in fact become “programmers.” Nothing is more important for the success of statistical software than enabling this transition from user to programmer, and on to gradually more ambitious software design. What does the user need? How can the design of statistical software help? This article presents a number of suggestions based on past experience and current research. The evolution of the S system reflects some of these opinions. Work on the Omegahat software provides a promising direction for future systems that reflect similar motivations.     

Unsupervised Learning With Random Forest Predictors

A random forest (RF) predictor is an ensemble of individual tree predictors. As part of their construction, RF predictors naturally lead to a dissimilarity measure between the observations. One can also define an RF dissimilarity measure between unlabeled data: the idea is to construct an RF predictor that distinguishes the “observed” data from suitably generated synthetic data. The observed data are the original unlabeled data and the synthetic data are drawn from a reference distribution. Here we describe the properties of the RF dissimilarity and make recommendations on how to use it in practice.

An RF dissimilarity can be attractive because it handles mixed variable types well, is invariant to monotonic transformations of the input variables, and is robust to outlying observations. The RF dissimilarity easily deals with a large number of variables due to its intrinsic variable selection; for example, the Addcl 1 RF dissimilarity weighs the contribution of each variable according to how dependent it is on other variables.

We find that the RF dissimilarity is useful for detecting tumor sample clusters on the basis of tumor marker expressions. In this application, biologically meaningful clusters can often be described with simple thresholding rules.