A new interaction force decomposition maximizing compensating forces under physical work constraints

In manipulation tasks interaction forces are often decomposed to be able to control robustness-reflective and accelerating forces separately. While this decomposition is typically performed for the synthesis of interaction forces to be applied for example in the context of robotic grasping, less attention has been paid to the analysis of measured, human interaction forces. Here, we introduce a physically-motivated bounding constraint, based on the law of energy conservation, and present a new decomposition approach for interaction force analysis with rigid objects. The decomposition extends the intuitive solution known in literature for the two finger grasp by maximizing robustness-reflective forces while respecting the bounding constraint. Advantages of our approach are illustrated in numerical examples and experiments and by comparing it to existing decomposition approaches. In contrast to existing approaches, our new approach is not limited in the number of interaction points and incorporates only individual interaction forces which are physically plausible.


I. INTRODUCTION
Grasping, as a frequently used and complex skill, has caught attention in robotics since the 70's.In general, the grasping task involves manipulation of an object by applying task-dependent and multi-purpose interaction forces that accelerate or deform the object.Consequently, interaction forces (IFs) can be decomposed into compensating forces (CFs), also called grasping forces, and manipulating forces (MFs).A CF is the component of an IF which has, combined with the other CFs, no effect on the acceleration of the object.They rather introduce stability and robustness to the grasp.An MF, on the other hand, is the component of an IF, which accelerates the object.This composition of IFs is used in robotic grasping for IF generation, also called IF synthesis [1], [2].In contrary, we aim for the decomposition of measured IFs, also called IF analysis, which is of great interest in a series of research areas such as the analysis of forces involved in joint object manipulation [3]- [5] and human grasp analysis [6]- [10].
In case of IF analysis, the decomposition of given IFs into CFs and MFs requires solving an under-determined system of equations.Thus, a meaningful solution has to be found from the infinite number of possible solutions by making additional assumptions and thus, reducing the solution space.In contrast, IF synthesis requires the composition of IFs from CFs and MFs, which have been derived based on requirements, e.g.grasp robustness studied by Aicardi et al. [11].For robotic grasping IF analysis and synthesis are used simultaneously: The measured IFs are decomposed using IF analysis to be able to calculate an error, while the IF reference values are determined using IF synthesis.
Yoshikawa and Nagai [12] used intuitive constraints to determine CFs and MFs from given IFs for two, three and four finger grasps.
Williams and Khatib [13] introduced the Virtual Linkage Model and solved the under-determined system using the Moore-Penrose Pseudoinverse which leads to the solution with the smallest norm.
Bicchi [14] detailed the composition of IFs and introduced a calculation scheme for the decomposition of forces during whole body manipulation that incorporates body parts like wrist, elbow or hip.He describes the CFs as a sum of active (corresponding to controllable system modifications) and passive CFs (corresponding to uncontrollable system modifications) using the manipulation stiffness and manipulator Jacobian.Then, the IFs can be determined using IF synthesis maximizing, for example, grasp robustness.
An approach for IF analysis, which allows subspace dimension calculation of controllable and uncontrollable parts of CFs and MFs was developed by Zhang and Gruver [15].They classified grasps into three categories: power grasps, constrained motion grasps and free motion grasps [16].
While the approach of Yoshikawa and Nagai [12] has the disadvantage that it abstracts the interaction to points and, thus, allows no torques to be applied on the object, the virtual linkage model of William and Khatib [13] includes also torques, but suffers from singularities at certain configurations.As will be shown later, the virtual linkage model further incorporates virtual forces, i.e. physically implausible forces, while methods not generating such virtual forces have been presented by Zhang et al. [15], [16] and by Yoshikawa and Nagai [12].However, the approach of Zhang et al. allows no direct calculation of CFs and MFs but rather the computation of controllable and uncontrollable parts.Furthermore, Yoshikawa and Nagai present decomposition approaches for two, three and four finger grasps only and thus, it remains unclear whether a solution for more than four fingers exists.Even more, as will be shown, the decomposition for three and four finger grasps results in few feasible decompositions or even an empty set for many grasp configurations.
In this paper, we introduce a physically-motivated bounding constraint, based on the law of energy conservation.This allows to extend the approach of Yoshikawa and Nagai, resulting in an optimization problem, which maximizes robustness-reflective CFs, is suitable for all numbers of contact points and only incorporates physically plausible IFs.

A. Problem Formulation
We consider a rigid object and point contacts, which means that only interaction forces, but no interaction torques may be applied to the object.However, the interaction forces may induce torques resulting on the object.For clarification Fig. 1 shows an object with an object-fixed coordinate system C, vectors r i pointing from the origin of the coordinate system to the respective contact points and interaction forces f i .The effect of an IF i on the object can be twofold and thus, Fig. 1: Object interaction with three interaction points.can be decomposed as follows: If wrenches resulting from the IFs exist, which compensate each other, the object is squeezed, stretched or distorted and an internal wrench describing the mechanical stress inside the object, also called internal forces [17], evolves.We call components of the IFs with this property compensating forces (CFs) and refer to them with f c,i throughout this paper.If wrenches resulting from an IF are not compensated, they accelerate/decelerate the object and an external wrench describing the motion of the object, also called external forces [17], evolves.We call components of the IFs with this property manipulating forces (MFs) and refer to them with f m,i .The MFs generate a resulting wrench w r acting on the object with where f r is the accelerating force, τ r the accelerating torque and whereby I is the identity matrix and R i the skew symmetric matrix operator of r i performing the cross product.CFs, on the other hand, generate internal wrenches w c,i and following their definition they sum up to zero.It follows with Using (1), ( 2) and ( 3), an under-determined system of equations results.

B. Related Work
In literature, different solutions to this under-determined system have been proposed as detailed in the introduction.In the following paragraphs, we will have a more detailed look at two representative solutions, the Virtual Linkage Model of Williams and Khatib [13] and the more intuitively derived approach of Yoshikawa and Nagai [18].
For a better comprehension of the Virtual Linkage Model the example shown in Fig. 2 will be used with The general solution of the Virtual Linkage Model for the resulting MFs is which reduces to It can be shown that the solution to (4) is invariant to shifts of the object-fixed coordinate system and thus, after shifting the reference frame to i r i = 0, (5) can be considered a simplified solution of (4).From (5) it can be seen that the resulting force f r on the object is distributed equally on all MFs.For the example in Fig. 2 this means f m,2 = f 1 /2 = 0. Hence, even if an IF has no influence on the acceleration of the object, it is assigned an MF larger than zero.We call these forces virtual forces because they are physically implausible and therefore non-existent.Please note, however, that in manipulation tasks the MFs and CFs are always synthesized to IFs and these again to resulting forces and torques that act on the object.On this macroscopic level, virtual forces disappear and the law of energy conservation holds rendering object manipulation possible.Only when looking at the individual forces and their components virtual forces can be observed, which is problematic for IF analysis as these forces are the ones of main interest.The approach of Yoshikawa and Nagai in contrary does not lead to virtual forces, but has other drawbacks.Their method is based on three intuitive assumptions: First, CFs should always be inside the friction cone.Second, an MF should have no part pointing into the inverse direction of the corresponding CF.Third, an MF has no part resulting in compression or tension of the object, neglecting torsion.From these assumptions, follow two steps for IF decomposition.In the first step, possible grasp modes α with α i ∈ {−1; 1} have to be chosen.A grasp mode describes if CFs between two interaction points squeeze or stretch the object and depend on the surface normals and the friction coefficients at the interaction points.A grasp mode can be calculated for the three finger grasp by using the algorithm described in [12].The CFs are described in a subspace h c using these grasp modes.In this subspace, a solution is only feasible if all values are positive.Otherwise, paradoxically the grasp mode would define compression, while the subspace value would result in tension.In the second step, given a grasp mode, different solutions to the MFs, again described in an own subspace h m , are tested for feasibility, i.e. no MF results into tension or compression of the object neglecting torsion and no MF points into the inverse direction of its corresponding CF.The different solutions result from any perturbation of a selection 1} that selects possible directions for the MFs.From the above considerations, the following system of equations results: After calculating all solutions for the MFs from the perturbations of k, the feasible solution, if one exists, has to be found by testing if the subspace values fulfill the assumptions.Using the resulting h, the CFs and MFs can be calculated by This approach has multiple drawbacks.For example, it is possible that multiple grasp modes (see [12] for examples) and eventually multiple solutions exist or that no grasp mode exists.There may be also no selection vector, preventing a feasible solution and, thus, decomposition.This is due to the constraint requiring that an MF is composed of forces which do not lead to tension or compression for selected parts of the corresponding CF (compare Condition 3 for MFs in [12]), which also reduces the solution space to the empty set for most grasp configurations with four fingers.Furthermore, only algorithms are given to decompose two, three and four finger grasps, because complexity increases drastically when adding additional interaction points.Thus, it remains unclear if a solution for more than four fingers exists.

C. Proposed IF Decomposition
So far, the mathematical decomposition into MFs and CFs is based on the definitions given in Section II-A.We will Fig. 3: The visualized solution space given by (6).Every vector combination consists of f m,i and f c,i .
extend these definitions to allow only physically plausible individual interaction forces by introducing the following bounding constraint.
Bounding constraint: This constraint is inherently included in the verbal definition of the MFs, but has not been motivated in literature yet to the best knowledge of the authors.Because an MF is the part of an IF, which accelerates the object, it also does physical work resulting in a differential change in energy of the object.Thus, by the law of energy conservation, the differential change in energy resulting from the MF cannot be larger than the one resulting from its corresponding IF.This is stated in Lemma 1 (see appendix for proof).

Lemma 1. Considering that an MF is the part of its IF, which performs physical work and taking (1) into account it follows
Remark: The inequality constraint (6) bounds the solution space for the CFs and the MFs to a sphere around their respective IFs with radius |f i |/2 as illustrated in Fig. 3 for two dimensions.Any violation of this inequality violates the law of energy conservation.
Given the bounding constraint, the solution space is reduced to physically plausible forces, but still an infinite number of solutions exist.A first intuitive approach to solve this problem would be to define that MFs contribute only to the resulting wrench on the object without any compensating parts, which we define as full decomposition.This constraint cannot be fulfilled for most manipulation situations and, thus, full decomposition would lead often to the empty set.This finding is formulated in Lemma 2 (see appendix for proof).

Lemma 2. Full decomposition of IFs into
where e 12 represents the unit vector from one interaction point to the other.In (7) the interaction forces are projected on the line connecting the interaction points, which is also done in the VL model, and the smaller projected force is chosen as compensating component.This is due to the fact that both CFs have to compensate each other, which means that their norms have to be equal and thus, only the smaller norm can be fully compensated.In other words, the CFs are maximized which is the first property we abstract from this approach.Extending this idea to multiple fingers, we thus propose to design a cost function that maximizes the CFs.However, the solution space of the CFs has to be bounded so that they do not increase to infinity.In the approach by Yoshikawa and Nagai, this is achieved by the proposition that an MF should have no part pointing into the inverse direction of the corresponding CF and vice versa, i.e. 0 ≤ f T m,i f c,i .To the best knowledge of the authors this constraint has not been physically motivated in literature yet.Using standard mathematical operations one can see our newly introduced bounding constraint ( 6) is equal to the constraint of Yoshikawa and Nagai.For the first time we motivate this constraint based on an energy conservation argumentation for the individual interaction forces.This constraint is finally used to restrict the previously introduced optimization.A last property is given by the fact that CFs can only be applied along the line connecting the two interaction points, which is the only solution to (3) for the two finger grasp [19].Hence, using (3) additionally to the bounding constraint allows us to expand the approach of Yoshikawa and Nagai to any number of fingers by formulating an optimization problem.

IF Decomposition Theorem (IFDT). For a precision grasp, the IF decomposition problem is given by the following optimization problem:
This optimization problem is also applicable to pulling forces.The formulated problem, however, is concave, because it is equivalent to minimizing that the optimization could result in local minima [20].Thus, online computation brings some difficulties.Please note that inequality (11) is equal to (6) (for details see proof of Lemma 1).
Remark: It can be shown that the solution for the two-finger grasp equals the solution proposed by Yoshikawa and Nagai as shown in Lemma 3 and as proven in the appendix.

III. NUMERICAL EXAMPLES
For illustration and comparison of the new decomposition approach with state-of-the-art approaches, namely the ap- x y Fig. 4: A three finger grasp.proach of Yoshikawa and Nagai [12] and the Virtual Linkage Model [13], the example of Fig. 4 is adopted with and some ε ∈ R. Values in z-direction are assumed to be zero and it should be noted that r 1 + r 2 + r 3 = 0 so that (5) can be used directly.The decomposition based on the newly proposed IFDT was performed using the optimization toolbox of MATLAB adopting an interior-point algorithm, which is suitable for quadratic optimization problems with nonlinear equality and inequality constraints.Assuming ε = 0, f 2 and f 3 are compensated by f 1 and the only force influencing the objects motion is the x component of f 1 .The MFs and CFs of the Virtual Linkage Model (vl) can be determined using ( 5) and ( 1) and are given by: vl f m,1 ≈ 0.67 0 T , vl f c,1 ≈ 0.33 2 T , vl f m,2 ≈ 0.17 0.29 T , vl f c,2 ≈ −0.17 −1.29 T , vl f m,3 ≈ 0.17 −0.29 T , vl f c,3 ≈ −0.17 −0.71 T .(6) and, thus, for vl f m,2 virtual forces are calculated.Also, vl f m,3 would contain virtual forces if the influence of f 1 on the object's motion gets larger (compare (5)).Summarizing, for the Virtual Linkage Model, we can conclude that an object accelerated mostly by a specific IF, with accelerating forces (MFs) much larger than the stabilizing forces (CFs), contains primarily virtual decomposed MFs.Using the approach of Yoshikawa and Nagai the grasp modes shown in Fig. 5 result.However, for ε [0.356; 0.577] there exists no solution because the constraint prohibiting the MFs to lead to tension or compression along a joining line cannot be fulfilled.Thus, there may always be grasp constellations preventing this decomposition.In contrast to these approaches, the IFDT leads to the following MFs and CFs, which are respecting the bounding constraint:

IV. EXPERIMENTAL RESULTS
We further performed a series of experiments to better illustrate the capabilities of the new approach and to compare results with state-of-the-art approaches.

A. Experimental Setup and Procedure
A cylindrical object with diameter 5 cm was printed using rapid prototyping and fixed to the end-effector of a haptic interface that was operated in admittance control mode and used to simulate an object with an inertia of 5 or 10 kg restricted to movements in a horizontal plane.The object was instrumented with either 3 or 4 Touchence force sensors of type TSSI OD10 FPC19 on the circumference of the object.Each force sensor was covered with a small hemispherical pad to reduce the contact between finger and force sensors to a point, see Fig. 6.A virtual environment was developed showing a horizontal circle and the subjects were asked to move on this circle in clockwise direction.Two experiments were considered to illustrate the applicability of the new force decomposition approach to the analysis of: 1) a three finger precision grasp and 2) human-human collaboration involving two interaction points for each human.Compensating and manipulating forces were analyzed for both scenarios using different force decomposition approaches. 1) Precision Grasp: Three force sensors were placed at an angular distance of 120 degrees on the circumference of the object, see Fig. 7, left.The subject was asked to grasp the object using a three finger circular precision grasp and to move the object clockwise along the circle shown in the virtual environment.A fast trial with 10 kg and a slow trial with 5 kg inertia were recorded.
2) Human-Human Collaboration: Four force sensors were placed at an angular distance of 120-60-120-60 degrees on the circumference of the object.Two subjects were asked to collaborate on the task of moving the object along the circle shown in the virtual environment.Each subject used a two finger precision grasp touching two adjacent force sensors, see Fig. 7, right.To avoid collisions with their hands, one subject grasped the object from the side, while the other subject grasped the object from above.Subjects were initially assigned the role of a leader and follower and instructed to alternately switch their role after every completed circle.In Fig. 8a a fast movement is shown where forces at the individual contact points reach their maximum when the movement direction is vertical to the force sensor surface.While this effect can be clearly seen in the plots for the MFs of the methods of Yoshikawa and Nagai and the proposed IFDT method, the Virtual Linkage Model averages this effect over all manipulating forces and thus, leads to virtual forces.Fig. 8b shows the same task, but now performed much slower.As can be seen, the results of the three methods are now much more similar.For the Yoshikawa method, however, no solution could be found for various points on the circle.
2) Human-Human Collaboration: Fig. 9 illustrates the results obtained for the human-human experiment.The method proposed by Yoshikawa and Nagai resulted in an empty set for all data points.Also for the Virtual Linkage model, no solution could be obtained as the four contact points were placed on a plane, and for such a configuration, the

IFDT -Compensating Force
Force 1 magnitude Force 2 magnitude Force 3 magnitude (b) Slow movements, 5 kg of inertia Fig. 8: Manipulating and compensating forces for the precision grasp experiment virtual linkage is at a singularity and consequently the grasp description matrix becomes deficient, see [13] for more information.In contrast, the IFDT scheme provides a physical plausible decomposition during offline computation.This clearly demonstrates the superiority of the proposed IFDT scheme in this case, where it comes to more than three interaction points.
In the top figure, the switching roles between the users can be observed.While in the upper half of the circle, user one (represented by interaction forces one and two) dominated the task, in the lower half of the circle, user two (represented by interaction forces three and four) dominated.Looking more carefully at the two forces recorded for each individual, one can see that the manipulating forces one and three dominated the task at specific angles on the circle.Analyzing these moments more carefully, it can be observed that the manipulating forces reach their maximum at points when the particular interaction force is applied tangential to the circle, allowing to use the whole force for accelerating the object.Corresponding results with a dominating finger two and four would be obtained for the reverse order of roles or the inverse direction of movement.

V. CONCLUSION
We have introduced a new approach for the decomposition of IFs into MFs and CFs for IF analysis.For this purpose, an intuitive approach originally introduced by Yoshikawa and Nagai for the two finger grasp has been formalized

IFDT -Compensating Force
Force 1 magnitude Force 2 magnitude Force 3 magnitude Force 4 magnitude Fig. 9: Manipulating and compensating forces for the humanhuman collaboration experiment and extended to more than two interaction points resulting in an optimization problem, which maximizes CFs.This maximization is only possible due to a new introduced bounding constraint, which bounds the solution space of the MFs and the CFs.The constraint is motivated by considering that a force component cannot do more physical work than the original interaction force.
Existing decomposition approaches were compared to the newly proposed IF decomposition.The Virtual Linkage Model was found to lead to implausible MFs when taking into account the law of conservation of energy for each interaction force separately and suffers from singularities at certain configurations.In contrast, the drawbacks of the method of Yoshikawa and Nagai were found to be that solutions are currently restricted to two, three and four interaction points, and that depending on the specific configuration, no decomposition can be obtained, specifically for grasps with more than three interaction points.
Since the optimization problem introduced in our IFDT method contains a quadratic constraint which complicates an analytical solution and focuses on forces only, our future work will target a numerical solution for online decomposition of IFs as well as its extension to include torques.

Lemma 1. Considering that an MF is the part of its IF which performs physical work and taking
Proof.Because an MF is the part of its IF contributing to the accelerating wrench acting on the object, two properties can be formulated: First, the projection of the IF on its corresponding MF can be used to describe an upper bound for the physical work performed by the MF.Second, if the work performed by the IF is positive/negative, also the work done by the corresponding MF must be positive/negative.From these statements, the following two inequalities can be formulated which must hold for every infinitesimal line segment dr = rds with |r| = 1 0 or rotary segment dφ = qdϕ with |q| = 1 0 Substituting ( 13) into ( 14) and (15) yields Comparing the coefficients it follows Substituting c i into (16) leads to Since f T i f m,i ≥ 0 the angle α between f i and f m,i is within [−π/2; π/2].When substituting (1) into (17) follows Using standard mathematical operations, we can get (12) from either (17) or (18) when assuming that (1) holds.

Lemma 2. Full decomposition of IFs into CFs leading to wrenches compensating each other and MFs contributing to the resulting wrench only, i.e. without compensating parts, is in general not possible.
Proof.Assume that the wrench w m,i , exerted by an MF on the object, is not pointing into the direction of w r .Then, the deviation between w m,i and w r must be compensated by another wrench w m,j where j = i, contradicting full decomposition.Consequently, all w m,i must point into the direction of w r .This can be formulated mathematically, when describing the MFs with two parts: one part pointing into the direction of f r and a second part, denoted as f x,i , leading to a torque aligned with the direction of τ r .
TABLE I: Possible sets of α for given projections of the IFs on the line connecting the interaction points.In (20), the torque on the left side is orthogonal to r i , while the torque on the right side is aligned with the direction of τ r .Thus, a solution to (20) exists only if τ r is orthogonal to r i .It is easy to find an example which contradicts this constraint and, thus, it can be concluded that in general, the MFs cannot have only parts contributing to the resulting wrench, but will also contain compensating wrenches.
Lemma 3. The solution to the two-finger grasp obtained with the newly proposed IFDT equals the intuitive solution (7) derived by Yoshikawa and Nagai [18].
Proof.For the two-finger grasp constraint (10) can be written as follows Assume that f T 1 e 12 > 0 and f T 2 e 12 < 0 and note that the left sides of (24) and (25) are always positive or zero.Then, if α ∈ R − 0 , (24) and (25) allow the solution α = 0 only.On the other hand, if α ∈ R + 0 , the cost function can take positive values.Similar considerations can be made for all other combinations of signs of f T 1 e 12 and f T 2 e 12 , summarized in Table I.Hence, dependent on the IFs and the line connecting the interaction points, one out of four possible solutions exists for maximization of (23) under the constraints (24) and (25).Two of them require α = 0.The other two can be determined by reformulating (24) and (25) as follows with k 1 , k 2 ∈ R + 0 .If f T 1 e 12 > 0 and f T 2 e 12 < 0, α is positive and must be maximized under constraint (26).Thus, α equals either k 1 or k 2 .Second, if f T 1 e 12 < 0 and f T 2 e 12 > 0, α is negative and must be minimized under constraint (27).Thus, α equals either −k 1 or −k 2 .Combining these results, we can state that α = ± min(|f T 1 e 12 |, | − f T 2 e 12 |).

Fig. 6 :
Fig. 6: Experimental setup showing the instrumented object mounted at the end-effector of a haptic interface and virtual environment (lower right corner)

Fig. 7 :
Fig. 7: Subjects holding the object in the precision grasp (left) and human-human collaboration experiment (right) B. Results1) Precision Grasp: Fig.8illustrates the obtained force decomposition results for the Virtual Linkage Model, the method proposed by Yoshikawa and Nagai and the newly proposed IFDT method.The graphs on top illustrate the direction and amplitude of the IFs measured at the individual contact points when moving in clockwise direction along the virtual circle.The length of the vectors corresponds hereby to the amplitude of the IFs.Below, manipulating and compensating forces are shown separately for each method.In Fig.8aa fast movement is shown where forces at the individual contact points reach their maximum when the movement direction is vertical to the force sensor surface.While this effect can be clearly seen in the plots for the MFs of the methods of Yoshikawa and Nagai and the proposed IFDT method, the Virtual Linkage Model averages this effect over all manipulating forces and thus, leads to virtual forces.Fig.8bshows the same task, but now performed much slower.As can be seen, the results of the three methods are now much more similar.For the Yoshikawa method, however, no solution could be found for various points on the circle.2) Human-Human Collaboration: Fig.9illustrates the results obtained for the human-human experiment.The method proposed by Yoshikawa and Nagai resulted in an empty set for all data points.Also for the Virtual Linkage model, no solution could be obtained as the four contact points were placed on a plane, and for such a configuration, the

)
From the first row follows that the two CFs have equal norms and point in opposite directions f c,2 = −f c,1 .Inserting this result into the second row of (21) and rewriting the cross product in its original form gives(r 1 − r 2 ) × f c,1 = 0 with the trivial solution f c,1 = f c,2 = 0 or f c,1 and f c,2 parallel to r 1 − r 2 .The non-trivial solution means that both CFs lie on the line connecting the two interaction points, which can be parametrized using e 12 from (8):f c,1 = αe 12 , f c,2 = −αe 12 α ∈ R.(22)Thus, the value α remains to be determined.Using (22) the optimization problem (9)-(11) can be reformulated to arg maxα J = 2α 2(23)s.t.α 2 ≤ αf T 1 e 12 , (24) α 2 ≤ −αf T 2 e 12 .