Human sit-to-stand transfer modeling towards intuitive and biologically-inspired robot assistance

Sit-to-stand (STS) transfers are a common human task which involves complex sensorimotor processes to control the highly nonlinear musculoskeletal system. In this paper, typical unassisted and assisted human STS transfers are formulated as optimal feedback control problem that finds a compromise between task end-point accuracy, human balance, energy consumption, smoothness of motion and control and takes further human biomechanical control constraints into account. Differential dynamic programming is employed, which allows taking the full, nonlinear human dynamics into consideration. The biomechanical dynamics of the human is modeled by a six link rigid body including leg, trunk and arm segments. Accuracy of the proposed modelling approach is evaluated for different human healthy and patient/elderly subjects by comparing simulations and experimentally collected data. Acceptable model accuracy is achieved with a generic set of constant weights that prioritize the different criteria. Finally, the proposed STS model is used to determine optimal assistive strategies suitable for either a person with specific body segment weakness or a more general weakness. These strategies are implemented on a robotic mobility assistant and are intensively evaluated by 33 elderlies, mostly not able to perform unassisted STS transfers. The validation results show a promising STS transfer success rate and overall user satisfaction.


Introduction
The rapidly ageing society and the continuous decrease of nursing specialists call for new assistive devices that fit elderly and patient demands.Human sit-to-stand (STS) transfers are a frequently exercised daily activity, which highly influences the quality of life of people who are not able anymore to accomplish normal STS transfers due to a specific or more general muscle weakness.
Only few assistive robotic devices focused on supporting human STS transfers so far and their control can be grouped into three categories: motion control, force control and switching control.
The group of Hirata and Kosuge presented different types of walking helpers that use basic admittance control to decide on the motion of the platform during STS transfers (Chuy et al. 2006).In Jun et al. (2011) and Kim et al. (2011) authors guided sit-to-stand transfers by a selected trajectory of a support plate mounted on the developed robot called SMW.The desired trajectory of this plate was implemented by controlling the linear actuator guiding the angle and height of the support plate.The authors proposed two predefined trajectories and compared their characteristics using the force/torque data measured by sensors at the top plate.Médéric and Pasqui finally developed a mobility assistant equipped with 2 degree of freedom actuated handles that support patients in STS transfers, see Mederic et al. (2004).They fitted pre-recorded hand paths with cubic splines and tested a series of preparametrized trajectories (Pasqui et al. 2010).
Force control was employed by Médéric and Pasqui who evaluated the zero moment point (ZMP) for a simplified human model and controlled the interaction force between user and robot to stabilize the configuration.They solved an optimization problem that minimizes the interaction force taking ZMP-based balance constraints into account (Mederic et al. 2005).
Finally, also switching controllers were investigated.A STS rehabilitation system consisting of a 3 DoF support pad that the patient must lean on during STS transfers was proposed in Chugo et al. (2008).Analyzing the different phases of STS movements by means of multi-body computer simulations, they realized an admittance controller with force reference implementing damping control for the lifting body phase and compliant impedance control for all other phases with a pre-computed reference trajectory based on real human STS transfers (Chugo et al. 2012).Pasqui et al. (2007) presented a fuzzy controller to ensure stability of the patient during assisted sit-to-stand transfers.They subdivided the sit-to-stand transfer into several phases and defined fuzzy rules that evaluate the center of pressure and the horizontal component of the handle force to guarantee stability for the patient by switching between controllers implementing variations of admittance control.
Understanding and imitating the human behavior during STS transfers provides a powerful tool to control assistive robots when aiming for an intuitive and natural behavior of the coupled system of human and robot.Previous work on human STS transfer assistance hardly incorporates computational models of STS transfer motions.STS transfers are mainly studied and analyzed in explorative and hypothesisdriven experiments, which led to a considerable amount of findings.Lindemann et al. (2003) for example developed a correlation formula to derive required power from body weight and standing up duration.Kralj et al. (1990) studied different STS transfer phases and their duration.Schenkman et al. (1990) divided STS transfers into 4 phases and discussed characteristics of these phases.STS transfer characteristics such as torque and range of motion in the lower limb joints during a normal STS were reported by Lindemann et al. (2003), Galli et al. (2008), Kralj et al. (1990).Further, Janssen et al. (2002) summarizes seat, subject and strategy-related determinants for STS transfers and describes their influence on performance.In Millington et al. (1992), Kotake et al. (1993), Ikeda et al. (1991), Hirshfeld et al. (1999) authors study average time, maximal hip flexion, knee extension angle and velocities for completion of a STS transfer.Modifications of CoM trajectories during STS transfers by lowering the horizontal and vertical CoM displacements were found to lead to a significant reduction of joint moments on the knee and hip, see Mathiyakom et al. (2005).Shifting the chair height from 65 to 115 % of knee height resulted in a large change of moments in hip and knee joints, see Rodosky et al. (1989).Moreover, minimum peak joint moments and their relation to movement time were determined by studying a large set of experimentally collected kinematic data in Yoshioka et al. (2007Yoshioka et al. ( , 2009)).More findings are summarized in reviews like Kerr et al. (1991) and Janssen et al. (2002).
While this way a huge variety of data has been analyzed by various researchers, only few computational models to study human STS transfers have been presented so far.In Mughal and Iqbal (2005) and Bahrami et al. (2000) authors investigated an optimal LQR formalism in the context of an optimal tracking controller combined with a fuzzy biomechanical model, which interpolates between two linearized models of the nonlinear four segment/bipedal dynamics around the sitting and standing position.They optimized physiological costs when tracking a predefined ankle, knee, hip, and pelvis reference trajectory (Mughal and Iqbal 2005, 2006, 2008a, b;Mughal et al. 2011).
In Mombaur (2014) the author employed a cost function combining joint torques squared with absolute head orientation.The author argues that the first term increases efficiency of the motion, while the second term results in a stabilization of the head, but no comparison with human data is performed allowing to judge whether this model is sufficient or appropriate to model human behavior in STS transfers.The same cost function has been adopted in Hoang and Mombaur (2015) for the design of a STS mechanism.
In Kuzelicki et al. (2005) authors employed dynamic optimization to determine optimal STS trajectories by considering a cost function that minimizes joint torques, torque change and the difference between left and right ground reaction forces based on sequential quadratic programming (SQP).They determined different weights of the single criteria for unassisted STS transfers of healthy subjects as well as amputees, but did not study assisted STS transfers.Moreover, critical balance criteria were not considered in their approach.Further, from an optimization point of view, SQP is considered a method of local optimization and thus, may lead to suboptimal solutions, while global methods based on the Hamilton-Jacobi-Bellmann equations and dynamic programming typically suffer from the curse of dimensionality.Both is problematic when considering biomechanical problems, as they are typically highdimensional and involve model uncertainties (Todorov and Jordan 2002).Differential dynamic programming (DDP) and iterative linear-quadratic Gaussian (ILQG) have been proposed in literature to overcome aforementioned limitations.They solve the optimization problem by dynamic programming, and lead to feedback control laws.Both are methods based on optimal feedback control (OFC) that have shown to be a powerful tool to study biological movements and interpreting human motor behavior (Todorov and Jordan 2002).
In this paper we formulate first unassisted and then assisted STS transfers as optimal feedback control problems and solve them using an iterative optimal control approach to derive optimal assistive strategies to be provided by an assistive robot.Optimal assistive strategies for subjects characterized by a specific or more general muscle weakness are studied, and optimal trajectories are derived.We employ DDP that iterativelly quadratically approximates the nonlinear system dynamics and the optimal cost-to-go function around the current trajectory.It takes physical control constraints like torque limitations into account, while human balance-related criteria are considered in the cost function.The modelling approach for unassisted STS transfers is validated for three different human healthy subjects and nine elderly/patient subjects by comparing simulations and experimentally collected data.
Finally, the STS model has been used to determine userspecific optimal assistive trajectories for 33 elderlies, mostly not able (or hardly able) to perform unassisted STS transfers.The obtained trajectories have been implemented on a robotic mobility assistant and intensively tested by the same subjects in a formal user study.The validation results show a promising success rate of achieved STS transfers.
This paper is organized as follows: STS transfer modeling is formulated as optimal feedback control problem in Sect. 2. Section 3 reports on capturing of experimental data, evaluates the model and compares simulation with experimental results.Section 4 presents obtained optimal STS assistive trajectories for different subject classes as well as results of the performed user study with the mobility assistance robot.Section 5 finally concludes the paper and presents some final remarks.

STS transfers formulated as optimization problem
In the following subsections the STS transfer task is formulated as an optimal feedback control problem with a nonlinear cost function subject to control constraints.An approximative optimal control approach based on DDP [firstly introduced by Mayne (1966) and recently reformulated by Tassa et al. (2012)] is employed to allow for an efficient solving of this optimization problem.

Human biomechanical model
While a triple inverted pendulum has been widely studied as a simplified biomechanical model of the human in biomechanics and biomedical literature [e.g.Iqbal and Roy (2004)], in this paper a model consisting of five joints and six rigid bodies1 involving foot, lower leg (shank), upper leg (thigh), trunk (torso and head), lower and upper arm is considered, which moves in the sagittal plane as shown in Fig. 1.The ankle, knee, hip, shoulder and elbow joint torques are used to control the motion of the model.The equations of motion are derived using the Euler-Lagrange method.The nonlinear dynamics of the biomechanical model is given by where M(θ ) ∈ R 5×5 is the positive definite symmetric inertia matrix, C(θ , θ ) ∈ R 5 the vector of Coriolis and centripetal forces, and G(θ ) ∈ R 5 the gravitational force vector, while θ ∈ R 5 refers to the joint angle vector with ankle (θ 1 ), knee ( θ 2 ), hip (θ 3 ), shoulder (θ 4 ) and elbow (θ 5 ) angles, τ ∈ R 5 the joint torques and τ ext ∈ R 5 the torque due to external assistive generalized forces applied to the human.The equations can be written as first order dynamic system with (2) Considering F ∈ R m external generalized forces applied to a specific point k on the human model, and (where m ∈ {1, 2, ..., 6}) the Jacobian associated to this point, then τ ext is given by Please note that in the unassisted case, we adopt a simplified version of this model controlled by three joint torques (arm segments not actuated).Moreover, in case of assisted STS transfers we study two different supporting points based on the level of the patient's demand advised by nurse specialists: (i) on the upper body under the patient's shoulders and (ii) at the hands.

Balance and task end-point accuracy criteria
To determine human balance and postural stability during STS transfers, the virtual zero moment point (for abbrevia-Fig.1 Rigid body biomechanical model of the human, l i and c i represent the length and center of gravity of the segments while x i ,y i are the reference frames attached to each joint tion ZMP) is evaluated.As summerized by Vukobratovic and Borovac (2004), the ZMP is a point on ground level where the pressure between the foot and ground is replaced by a force which can balance active forces acting on the human dynamics during the motion.ZMP can be computed from the vertical component of contact moment T and the horizontal component of contact force F as follows: Task end-point accuracy is determined using the center of mass (COM): where m i is the mass of the i th segment and pi the position of its center of gravity.

Formulation of optimization problem
The STS optimal control problem is formulated as follows: The human sitting position with zero joint velocities is considered the initial state at time t = 0 and the position of the COM in the steady-state standing position is considered the desired final state of the system at time t = T .The main goal is to find a control law τ * = π(x, t) that stays within joint torque limits and that drives the system states smoothly from the initial to the final configuration while minimizing a given cost function.
We consider three main features when defining the cost function of the optimization problem: user energy consumption, smoothness of motion and control as well as user balance.Minimization of energy is achieved by the effort term C 4 in (6) that tries to achieve a minimum time response and thus, a minimization of energy as joint torques are much lower in the standing than in the sitting configuration (when neglecting the interaction forces with the chair).Smooth control is achieved by the torque change term C 3 , while the jerk term C 2 improves smoothness of the resulting motion.As humans automatically try to stabilize their movement patterns, human balance criteria C 1 based on the ZMP are included as well. 2 The following combination of criteria is used to model the STS transfer task: with and W f 1 , W f 2 weighting matrices for the terminal costs evaluated at the desired human COM position p tar com in a standing position at time T with zero joint velocities, W 1 the weighting matrix for the human balance term that aims to satisfy p min zmp ≤ p zmp (x, τ ) ≤ p max zmp , 3 and 2 Please note that a precise study of the human balance behavior during a STS is out of focus of this paper, but is a very interesting biomechanical research question.Currently no study focusing on the balance criteria used during a human STS transfer that could inform the selection of these criteria could be found in literature and therefore regulation of the human ZMP position has been considered as a postural regulator as proposed by Li et al. (2011). 3The base of support (BOS), which determines the values of p min zmp and p max zmp ), typically includes the size of the feet and the room between them for a human without external support, respectively unassisted STS.For the assisted case, when the human firmly grasps the robot handles a larger BOS area can be considered.Since this, however, requires W 4 = diag(w 4a , w 4k , w 4h ) the weighting matrices for the human jerk, minimum torque change and effort terms respectively, where the term diag(.)represents a diagonal matrix 4 and |v| 2 W = v T W v. The weighting matrix W 5 is responsible for the human joint angle and velocity boundaries θ min ≤ θ(t) ≤ θ max and θ min ≤ θ(t) ≤ θ max .The weighting matrix W 6 is considered to minimize the interaction forces exchanged between assistive robot and human and therefore is considered equal to zero for the case of unassisted human STS modeling.The cost function is finally considered subject to constraints of the system dynamics formulated in (2) and control constraints, i.e. τ min ≤ τ (x, t) ≤ τ max .

Optimal feedback control
We solve this optimal control problem using Differential dynamic programming (DDP) first proposed in Mayne (1966) and recently reformulated by Tassa et al. (2012).This approach iteratively, quadratically approximates the costs and the nonlinear system dynamics around the current trajectory.Then, an approximately optimal control law is found by designing an affine controller for the approximated system that enforces formulated control constraints.More details are given in Appendix 1.
For our specific STS transfer problem we consider pure gravity compensating forces as an initial guess of the control sequence, which is then iteratively improved by the algorithm with respect to the formulated cost function.
The algorithm shows quadratic convergence in the vicinity of a local minimum, similar to Newton's method as presented by Liao and Shoemaker (1992) and returns the optimal control and the corresponding state sequences.

Inverse optimal control to determine cost function weighting factors
Deriving a proper set of weighting factors for the cost function is crucial to properly model human STS transfers.We employ an inverse optimal control (IOC) approach to identify underlying optimality criteria of STS motions either for healthy subjects or patients.IOC allows to identify unknown parameters in the cost function (in our case the weighting factors as defined in Sect.2.3) for a set of recorded human STS trajectories.We adapt the methodology proposed by Mombaur et al. (2010) to our specific problem of human STS motions.
Footnote 3 continued detecting whether the human stably grasps the handles and the current robotic platform is not equipped with proper sensors to do so, we decided to simplify the problem and to consider the most restrictive case defined by the BOS of the human user only. 4Please note that the same values for all diagonal elements are considered for each weighting matrix.
Given a set of recorded user STS motions, a cost function for the bilevel optimization problem is formulated as follows, min where the sum of the Euclidean distance between experimentally recorded states x M (t j ) and the results of the optimal control model x * (W , t j ) is used to determine optimal values for the weighting factors W .The bilevel optimization handles iterations over weighting factors such that the best fit between measurements and the solution of the original optimal control problem formulated in Sect.2.3 is found.
For each iteration the obtained solution of weighting factors resulting from the bilevel optimization problem is passed to the lower level where the original optimal control problem is solved and obtained results are reported back to the bilevel where ( 7) is evaluated for the next iteration.
We employed the Matlab fmincon Trust Region Reflective Algorithm solver to solve the bilevel optimization problem.Box constraints for each weighting factor were specified to define a search space for the solver.

User-group optimized STS assistance
Finally, we use the already introduced biomechanical model and optimization approach to calculate optimal assistive strategies for the robotic assistant that is used to support subjects in STS transfers.We implement assistive strategies that are tailored to the specific class and weakness of a certain subject.
In Dreben (2006) a classification scheme for transfer assistance was proposed that considers the request for supervision, type of assistance and participation of targeted persons.Here we focus on the two classes of maximal assist, "the patient contributes with less than 25 % of the required effort to accomplish the STS task ", and moderate assist, "the patient contributes with at least 50 % of the required effort to accomplish the STS task".As proposed by nursing specialists, the most common techniques for assisting persons in STS transfers belonging to the maximal assist class foresee that the caregiver stands in front of the person to be assisted, locks the knees and feet of the patient, grips the patient at the upper trunk and lifts the person.Stronger patients belonging to the moderate assist class require less physical assistance, but more balance support.In this case, the caregiver stands in front of the patient, grasps the hands and applies forces to assist in the STS transfer, while simultaneously assisting in keeping the patient's balance.
Moreover, the weakness may be either limited to specific segments of the body because of a certain disease or surgery (case a), or spread over multiple segments (case b).
For the maximal assist class, we considered that the required assistance is applied to the upper body under the patient's shoulders.For the moderate assist class, the interaction point is considered on the human hands.By solving the aforementioned optimization problem we determine optimal assistive strategies in form of robot motion trajectories.Doing so, we consider torque constraints in the optimal control problem, which are based on the level of the weakness in human segments (as discussed above), and constraints on the assistive forces to be applied at the contact point(s).
The accuracy and usefulness of obtained assistive strategies highly depends on the proposed human STS model, which has to be carefully validated.Therefore, in the following sections we study the validity of the proposed model, first for healthy subjects and then for elderly and patients.

Validation of STS model
In order to determine weighting factors and test the quality of the STS model against real measurements, we performed a set of STS transfer experiments with healthy and elderly subjects, where both cases of unassisted and assisted STS transfers were studied.In the following sections, we report on the validation methods and obtained results (Fig. 2).

Data capturing
Capturing of STS transfer motions has been performed in two sessions, first for healthy subjects and then for patients.

Healthy subjects
We performed STS transfer experiments with three healthy male subjects to test the quality of the STS model against real measurements.Their body measurements are listed in Table 3 of Appendix 2.
Participants were instructed to perform a few practice trials in order to find a comfortable feet placement.They were asked to keep their feet fixed to the ground, their arms crossed over the chest, and their upper body straight during the whole experiment (see Fig. 3).Each subject was asked to repeat five STS transfers at a natural speed while the seat heights were adjusted on an armless office chair to fit the lower leg length.Since a set of pre-performed experiments with different subjects showed that typically people leave the chair with upper-body inclination of about 30 • (where zero represents the upright trunk position), the subjects were asked to start the STS transfer with about 30 • initial inclination in order to reduce the effect of not-modeled chair support and therefore allow for a fairer comparison of model and experiments.
An Xsens MVN inertial motion capture system, see Roetenberg et al. (2013), was used for full-body human motion capture.The subjects were asked to wear the Xsens MVN motion capture suit which consists of MTx miniature inertial measurement units with 3D linear accelerometers, 3D rate gyroscopes and 3D magnetometers.These trackers are placed at strategic locations on the human body (in the suit), to measure motion of the whole body including 23 segments (22 joints).The accuracy of the Xsense system is highly dependent on the calibration procedure, where subjects are asked to keep their body in some predefined configurations.The calibration has been performed for every subject before starting recordings.In the performed experiments a very good visual observable accuracy was achieved since the healthy subjects could easily keep their body in the requested configurations, but as no ground truth with an external tracking system was available no absolute numbers for the achieved accuracy can be given.Kinematic data including segment position and orientation, velocity and acceleration were captured with a sampling rate of 120 Hz.
Every STS transfer was assumed to start from the static configuration in the sitting position and to finish when the user arrives at the fully standing position with zero joint velocities.The average STS transfer movement time for all subjects was found to be in the range of 1-2.5 s.

Elderly subjects
Validation of the proposed STS model was performed using a set of recordings from nine elderlies with varying age and gender (5 male and 4 female, from 75 to 87 years old) performing unassisted and assisted STS transfers.The recruitment of the participants was decided based on their cognitive and motor status.In general we targeted subjects with mild to moderate impairment levels.
Subjects were included if they met the following criteria: (1) unable to stand up and sit down unassisted from a normal chair or 5-chair stand test (Guralnik et al. 1994) >16.7 s and (2) habitual gait speed <0.6 m/s.The cognitive impairment was screened with the Mini-Mental State Examination (Folstein et al. (1975)) classifying patients into no, mild to moderate, or severe cognitive impairment if MMSE provides test scores ≥26, 17-26 or <17, respectively.The participant metadata consists of information about sex, age, height, weight as well as the cognitive and motor impairment level are listed in Table 4 of Appendix 2.
The experiments took place in the Agaplesion Bethanien Hospital/Geriatric Centre of the University of Heidelberg in November 2013, with the ethical approval by the Ethics review committee (Ethikkommission der Medizinischen Fakultät Heidelberg, Alte Glockengießerei 11/1, 69115 Heidelberg, Germany).
Two general variations of tasks were asked to be performed by patients in order to determine optimal policies regarding STS transfers: a. Unassisted STS transfers: three repetitions of STS transfers performed in the patient's own preferred way without providing any instruction for initial configuration, hand or feet positions.This was an optional task and was only performed if the patient was able to accomplish unassisted STS transfers.b.Assisted STS transfers: three repetitions of STS transfers when a passive rollator was positioned in front of patients and they were asked to grasp its handles to receive physical support while performing STS transfers.
The passive rollator was equipped with two 6 DOF force/torque sensors of type JR3 45E15 mounted on the rollator's handles.Figure 2 presents the snapshots taken during the unassisted and assisted STS experiments by one elderly.
Motion of the subjects was captured using a Qualisys system with eight infrared cameras mounted on tripods and placed around the recording area.A suitable marker set including 48 reflective markers was used to track the limb movements.The accuracy of the Qualisys tracking system depends on the type of camera chosen, the number of cameras used in the experiment, the size of the tracking environment, the selection of the marker sets and how the markers are fixed on the moving segments: In the performed experiments, the accuracy of 0.7 mm was achieved for tracking each of the markers.Markers were selected to result in least possible occlusions when tracking the whole body and based on suggested marker sets of the manufacturer of the system.However, the biggest source of inaccuracy (which should not be more than 1 cm, although we have no direct possibility to measure it) comes from the installation of markers on the subjects.Strips were used to fix the subject clothes in the vicinity of each marker to guarantee an as stable as possible marker position.Nevertheless, movements of the subject may have resulted in a slight shift of the markers with respect to the human skin.The usage of special stretchy clothes with free arms and legs was unfortunately declined by most elderly subjects and thus, the usage of strips was considered a good compromise.The motion capture data was post-processed in two steps: cleaning of raw data and labeling of marker trajectories using the QTM-manager software,5 reconstruction of the human model and extracting the motion data using Visual3D software.We used extracted human joint angles, velocities and accelerations for the computation of joint torques based on the human inverse dynamics.

Validation method
Captured data was preprocessed to remove noise using a lowpass filter with 5 Hz cut-off frequency.Parameters of the biomechanical model were estimated for all subjects using regression formulas provided by Zatsiorsky and Seluyanov (1983), see Table 1 for results of subject S1. 6 Based on the captured motion data, human torques and the COM trajectories were estimated based on the human inverse dynamics for each STS transfer motion.
Next, the proposed optimal control approach for simulating natural STS transfers was evaluated by comparing simulations with measurements.For each simulation, the same experimental conditions of initial upper body inclination and chair height as well as task completion time were considered.Average values for the weighting factors obtained for each user group were determined from the results achieved using the inverse optimal control approach, see Sect.3.3.2for more details, and then used for determining STS trajectories using the optimization approach described in Sect. 2. Finally, results of the optimization were compared with captured data from the instance where subjects left the chair as we did not consider the effect of the chair support in our optimizations.The joint configuration at this instance with zero velocity was used as initial condition for the optimization algorithm.

Weighting factors
Deriving proper weighting factors for the cost function is one of the most critical steps in order to achieve an acceptable STS modeling performance.An inverse optimal control approach was applied for each trial and subject to determine the corresponding weighting factors best fitting to the presented cost function for replicating human STS transfers.Only joint angles were required as input data for unassisted STS transfers, performed either by healthy or elderly subjects.For assisted STS transfers additionally the measured forces at the robot handles were taken into account.In order to determine the subject's weakness, the following steps were performed: first, required joint torques were computed based on the recorded motion data to determine required torques for a successfully performed STS transfer.Then, the recorded external force profiles were transformed into joint torques.Finally, the required assistance was estimated by relating Footnote 6 continued (in Sep 2015) a new study by Hoang and Mombaur (2015) proposed an adaptation of the parameters defined in De Leva (1996) specifically for elderlies.Using these adapted formulas may lead to a better estimation of the anthropometric data of elderly subjects.joint torques from the provided assistance to the total required joint torques for a successful STS transfer.
Since a STS transfer is a complex task, in the following section we investigate the effects of each weighting factor on the overall combination of different criteria used to model the STS motion.Moreover, we report on the obtained weighting factors using the bilevel optimization approach for each group of subjects.

Qualitative sensitivity analysis of weighting factors
To get an understanding for the sensitivity of results on varying weighting factors, each weighting factor was manipulated manually and the model accuracy was checked by comparing simulation results with obtained measurement data.Generally for all cases of experiments with healthy and elderly subjects, good modeling accuracy was obtained when the largest weighting factors were specified for the joint angle and velocity limits (W 5 ) to effectively remove unfeasible motions.To guarantee human balance (specifically for the case of assisted STS, see Sect.4), the corresponding weighting factors (W 1 ) need a high priority too, while lower values are required for the minimum jerk, minimum effort, and minimum torque change terms.For the minimum jerk term (W 2 ) small changes in its weighting factor were found to result in a rather high variation of simulated STS transfers.Reducing the value resulted in a relative high velocity impulse close to the end of the STS transfer and increasing the value resulted in smoother trajectories with a comparable deceleration at the end of the motion.Regarding the minimum effort term, smaller weighting factors of (W 4 ) for the knee compared to the hip and both smaller compared to the ankle resulted in a better modeling accuracy as the highest and lowest contribution for a STS transfer were observed to come from the knee and ankle, respectively.Concerning the minimum torque change term (W 3 ), low values of the weighting factor resulted in smoother motions and control profiles while larger values produced non-human like behavior.
Final term conditions (W f 1 , W f 2 ) in the cost function were also found to be a very important factor in the optimization.Selecting low values, no control in the sagittal plane was possible.On the other hand, very large values overruled all other factors in the cost function and thus, led to an immediate termination of the optimization as no improvement over  iterations could be achieved.We found that the body weight strongly influences the final STS model performance that required to consider these weighting factors in proportion to the user weight w.

Obtained weighting factors by inverse optimal control
Using inverse optimal control a series of weighting factors was finally determined for the cost function (6).Table 2 shows mean values of the obtained weighting factors for the three cases of healthy, unassisted elderly and assisted elderly STS transfers,7 while Fig. 4 also shows information on standard deviations.For the sake of presentation the values in Fig. 4 are normalized with the maximum values for each weighting factor found over all trials.Weightings for final terms as well as joint constraints are not shown since no variation (or only negligible variations) were found.Since the joint constraints have to be satisfied during the sit-to-stand motion, their corresponding weightings for the boundary conditions (W 5 ) were considered as constant large value for all cases and were removed from IOC.However, the corresponding weights for the final terms (W f 1 and W f 2 ) were found to be a function of the subject's total weight w as reported in Table 2. 8Focusing on the variation of the weighting factors, mean values of weighting factors W 1 and W 2 were found to be most similar between healthy and assisted elderly groups, while a very small variation was found for W 3 for all groups.According to the obtained weighting factors healthy and unassisted elderly subjects minimized more torque on the ankle than knee and hip (W 4,a > W 4,k > W 4,h ), see Table 2.However, no such prioritization was observed for the assisted elderly subjects.A considerable variation for different subjects was found in most of the weighting factors for assisted elderly subjects (maximum for W 1 and the lowest for W 3 ).

Validation results
Validation of the finally obtained models was performed by comparing simulations based on mean values of the weighting factors (reported in Table 2) and corresponding experimental results.The user's COM position and joint torques observed during STS transfers and averaged over the captured trials (five times for healthy subjects and three times for elderlies) were chosen for comparison of simulation and experiments.A first comparison showed that similar STS strategies were selected by different subjects in each group.
To provide a measure for the overall model accuracy, for all subjects the normalized integral of the error between experiments and simulation was computed as where v ex p and v sim refer to data in experiments and simulation respectively, with v ex p,max , v ex p,min the maximum and minimum value of experiments.This error was evaluated over the x and y components of the COM position (e com x , e com y ) and the ankle, knee and hip torques (e τ a , e τ k , e τ h ).The obtained results for all 3 cases of healthy, unassisted and assisted STS transfers are shown in Fig. 5.The maximum average error was obtained for unassisted elderly subjects in the ankle and hip torques.This is mainly due to the fact that most of the elderly subjects tried to benefit from external assistance using their hand to initially push their body up in unassisted STS transfers, see Fig. 2.This resulted in a small mismatch between the proposed STS model and experiments.Considering the complexity of the problem and the simplified assumptions for the human model, the errors in all three studied cases are considerably low and illustrate an overall high agreement of model and measurements.
As similar results were obtained for the performed repetitions in each group and for the sake of presentation we only present simulation and measurement trajectories for the COM position for five repetitions as well as joint angles and joint torques for one repetition in Fig. 6. 9 As can be seen all three joints as well as COM smoothly converge to their stable final configurations, which is well captured by the model in all cases.The initial errors for the user joint torques between simulations and corresponding experiments resulted mainly from neglecting the supportive chair effect, more specifically from neglecting initial subject velocities and accelerations at the instance of leaving the chair.A series of snapshots for above-mentioned STS transfers and corresponding simulation results are shown in Fig. 3 to further depict the similarity of the results.As can be observed the user leaves the chair while having almost 45 • upper body inclination. 9Please note that although subjects were asked to minimize variation, still non-negligible differences were observed, especially for initial upper body inclination and feet positions.

Robot-assisted STS transfers
In the following subsections we first report on optimization results obtained when taking into account external assistive forces.Then, we report on the realization of the proposed user-adapted STS transfer assistance with an assistive robot and a performed user study with elderly.

Optimization results considering external assistance
We implemented assistive strategies that are tailored to the specific class and weakness of a certain subject.We specifically report on simulation results for the two tailored assistive strategies of a maximal and moderate assist class as well as two patient categories with general or more specific muscle weakness (see Sect. 4).The weighting factors are considered as the mean value of the assisted elderly obtained from the model validation study and reported in Sect.3.3.2,while the weighting for minimization of assistive forces were considered equal to diag(8, 8, 8) for all assisted STS transfer simulations.The same values of diag(8, 8, 8) were considered for both cases to minimize the influence of this term on the STS transfer.The selection of these weighting factors is a design choice.In the paper weighting factors have been chosen to result in an overall smooth behavior.But different rationales may be followed for the selection of them: when for example aiming for mimicking human-human interaction weighting factors could be determined by replaying trajectories recorded in human-human interaction using the robotic system and recording the occurring interaction forces, which and weakness in specific joints (second row), as well as for patients belong to the class of moderate assistance and having equal weakness in all joints (third row) and weakness in specific joints (fourth row) (Color figure online) could then be used in the inverse optimal control to derive proper weighting factors.But also other principles may be followed to select weighting factors, e.g. based on a training program.

STS assistance for the maximal assist class
For the maximal assist class the required assistance is typically applied to upper body segments under the shoulders.The independent effects of the shoulder and elbow joints were neglected in the biomechanical model.This is mainly due the fact that the patients belonging to the considered maximal assist class are expected to perform very small hand motions during a STS transfer.However, the inertia effects of the arms are still considered in the upper body dynamics.We restricted the shank motion by considering a lower range of motion for the ankle joint (6) to mimic the influence of locking the patient's knees and feet during the STS transfer.
Considering the minimum torques needed to rise successfully without help, see Yoshioka et al. (2007), joint torque constraints (τ ankle < 75, τ knee < 75, τ hi p < 25) were assumed to simulate patients of case a and (τ ankle < 40, τ knee < 40, τ hi p < 40) for case b, respectively.We considered more than 50 % weakness for a person with body measurements similar to subject S1.We also activated the most effective interactive force components in the STS transfer model, namely vertical and horizontal external force components (F x , F y ) as well as the angular momentum (M z ). Figure 7 shows obtained trajectories of the assistive force/moment profiles to be applied to the user as well as the user's COM, joint angles and joint torques.As can be observed human weakness is compensated through proper external assistance as the STS transfer is smoothly accomplished while the human joint torque remained lower than the considered user's capability.

STS assistance for the moderate assist class
For the moderate assist class we assumed that the patient is able to rigidly grasp a robotic device that assists in STS transfers.Thus, also the human arm has been considered in the biomechanical model.Higher joint torque limits compared to the maximum assist class and the two sorts of weaknesses were set (τ ankle < 100, τ knee < 100, τ hi p < 50) for case a and (τ ankle < 55, τ knee < 55, τ hi p < 55) for case b.Required supportive force/momentum profiles as well as

User study
An intensive evaluation by 33 elderly subjects was performed to assess the effectiveness of the proposed optimal STS transfer assistance.Thirty women and three men participated in the evaluation which took place for six weeks (from Oct. to Dec. 2014), in the Agaplesion Bethanien Hospital/Geriatric Centre of the University of Heidelberg.The average age of participants was 84.5 ± 5.0, ranging from 74 to 94 years old.Subjects had on average moderate stage impairment in cognitive and motor status (MMSE score: 24.9 ± 3.9; 5-chair stand: 19.6 ± 3.9 s; gait speed 0.47 ± 0.13).Moreover, 64 % of participants experienced at least one fall within the past 12 months.All the users used normal walkers in their daily life.The experiments were performed with the ethical approval of the Ethics review committee (Ethikkommission der Medizinischen Fakultät Heidelberg, Alte Glockengießerei 11/1, 69115 Heidelberg, Germany).

Experimental setup
The assistive strategies have been implemented on a robotic mobility assistance platform equipped with two actuated rear wheels and two 2 DOF parallel actuated arms (see Fig. 8).Handle levers are designed to always keep the same orientation as shown in Fig. 8, and are equipped with a pair of 6 DOF JR3 force torque sensors.The arms are actuated using spindle drives controlled independently in the sagittal plane.
Each arm has two actuated revolute joints and one passive joint.The torques are applied to the actuated joints by linear actuators which are connected to the segments by the rotary joints.The torque at the passive joint is applied by a cable drive, which is rigidly connected to the first two actuated joints and keeps the handle in the horizontal position.The range of motion of each joint is as follows, 0 < θ 1 < 49 • and 0 < θ 2 < 91 • where θ 1 = θ 2 = 0 refers to the arm stretched in upright configuration, resulting in a reachable handle position of 0 < x < 0.5 m and 0 < y < 0.6 m in Cartesian space, where 0 refers to the handle position in The controller of the arms is implemented using MAT-LAB/Simulink Real-Time Workshop where the handle positions are controlled in Cartesian space using inverse kinematics and a high-gain low-level PID joint space position controller.Communication and sensing loop are set to run at T = 1 ms sampling time.
Because of workspace limitations of the used mobility assistance robot, we had to exclude very tall or small persons as the obtained optimal trajectories could not be realized.

Methods
Each subject performed a MiniMental test and its results were recorded along with body characteristics including patient's height, total weight and specific weaknesses.Anthropometric limb data for each patient was estimated based on weight and height information using regression formulas provided in Zatsiorsky and Seluyanov (1983).Each user was assigned to a specific target group according to their level of impairment and their specific or more general weakness were recorded.Consideration of the level of subject's impairment, advice of nurse specialists and the mean value of percentage of the weakness already used in Sect.3.3 allowed the selection of specific joint torque limits for the optimization.Finally, optimizations were performed to derive user-specific optimal robot handle trajectories taking the aforementioned joint torque limits into account.Because of robot workspace limitations, initial and final hand configurations were considered within the robot workspace to achieve optimal trajectories realizable with our assistive robot.The initial hand position was selected as close as possible to the subject hip sitting on a chair.An example of the achieved individualized robot handle trajectories is shown in Fig. 9.
During the experiments the chair height was adjusted to the user's knee height.Before testing the robot STS transfer assistance, the ability of participants to perform unassisted STS transfers was assessed.The participants were instructed to stand up without receiving any support neither from the assistive robot, nor from the nurse-specialist supervising the experiments.In order to standardize experiments, each subject was orally instructed how to use the assistive robot, where to sit and how to keep the feet position.Then, the assistive robot was placed in front of the patient and the robot handles were brought to the initial configuration for the STS task.The subject was asked to grip the robot handles and to trigger the robot controller by applying a rather small downward force, whenever they felt ready for the STS transfer.Each subject performed five STS transfers with the help of the assistive robot considering pauses to avoid exhaustion.After finishing all five trials participants were asked to fill a questionnaire.

Results
Figure 12 shows a sequence of the STS transfer assistance provided by the robot to an elderly subject.In total 165 STS trials were recorded from 33 participants.Apart from the results of the questionnaire (which will be reported in another manuscript reporting the clinical perspective), two performance metrics were considered in order to verify the effectiveness of the proposed STS transfer support: STS transfer success rates and similarity of the assistive force profiles in simulation and experiments.
All participants were able to stand up from the chair with the help of the assistive robot at least three out of five times, where all participants successfully achieved a standing position within the 4th and 5th trial.Figure 10 shows the average success rate recorded for all patients per trials.Across all participants and trials an average success rate of 93.3 % was achieved, while a success rate of 54.5 % was achieved for the preliminary test assessing the STS transfer ability without support.Subjects became quickly familiar with the robot and fast motor learning took place as the success rate increased quickly with trials.The 100 % success rate for the fourth and fifth trial shows a great success in providing robotic STS transfer assistance.Apart from the success rate for STS transfers, we aimed to compare the similarity of interaction forces between human and robot obtained in simulation and experiments.The forces and moments measured at each robot handle were transformed and summed as follows to define the total interaction forces and moments (F tot and T tot ) provided to the human, where h i i ∈ (l, r ) are the distance between the contact points (robot handles) and the center of robot handles.We mainly compare the horizontal and vertical components of the obtained force trajectories as well as the component of the moment orthogonal to the plane of the force axis.Although for many of the patients a good similarity has been obtained, see e.g.Fig. 11 (top), some of the subjects could not fully stretch their body in the final configuration and therefore a considerable high amount of assistive forces was required while standing, see e.g.Fig. 11 (below).As this latter case was not considered in our model, clear mismatches between measurements and simulation can be observed.To incorporate this effect joint limits need to be adjusted.Further, asymmetric motions as observed for patients with one-sided impairments can not be properly replicated by our 2D model and require an extension to 3 dimensions.
After completion of all five STS transfers, the subjective user perception was evaluated by means of a questionnaire adopted from Schwickert et al. (2011).High overall satisfaction with the optimal STS assistance system was observed.Details on this subjective evaluation though will be reported in another manuscript addressing the clinical perspective (Fig. 12).

Conclusion
We have presented an optimal feedback control formulation for the modeling of assisted and unassisted human STS transfers.Compared to previous work based on SQP approaches, we based our optimization on DDP that has been proven to be a powerful tool to study biological movements.It allows to obtain an optimal solution with respect to a defined cost function and considers the nonlinearity of the human biomechanics as well as physical constraints, which are naturally incorporated into the optimization framework.It further shows potential for future online implementation.
We showed that natural STS transfers could be achieved with the help of a cost function that linearly combines a series of factors and finds a compromise between task end-point accuracy, human balance, energy consumption as well as smoothness of motion and control and takes further human biomechanical control constraints into account.Validation of the proposed approach was performed for different healthy and elderly subjects.
The model was extended with external forces and torques and optimal assistive STS transfer strategies were determined considering two types of assistance classes and weaknesses.The resulting optimal assistive trajectories were calculated and implemented on a robotic mobility assistant.The assistive STS transfer approach was finally evaluated by 33 elderlies performing 165 trials.Results show a high user satisfaction as well as a 100 % success rate for all participants in the fourth and fifth trial.
Possible future extensions include the consideration of more complex three-dimensional human dynamic models with direct muscle control.Moreover, the inclusion of chair support forces, which will require switching the model during optimization, represent a natural extension of the presented approach.Ultimately, one may aim for online implementation of the optimal control approach for assisted sit-to-stand transfers.
approximates the costs and the nonlinear system dynamics the current trajectory.Then, an approximately optimal control law is found by designing an affine controller for the approximated system that enforces formulated control constraints.For our specific STS transfer problem we consider pure gravity compensating forces as an initial guess of the control sequence, which is then iteratively improved by the algorithm with respect to the formulated cost function.The iterative approach is implemented as follows: At each moment k, the cost for the optimal control of the system from the current state x k to the final state x N is defined by: where τ * i is the optimal control decision.This local approximation of the original optimal control problem can then be efficiently solved by evaluating the Hamilton-Jacobi-Bellman equation where Minimizing the right side of ( 13) with respect to δτ k determines the optimal control policy as follows, The resulting control law is of affine form δτ k + τ (i) k , and then the new nominal state and control trajectories are computed using the dynamic equations of the system.10

Appendix 2
The body measurements of the healthy and elderly participants in the validation of the STS transfer model (Tables 3,  4).

Fig. 2
Fig. 2 Snapshots taken during the unassisted and assisted STS experiments performed by elderly

Fig. 5 Fig. 6
Fig. 5 Normalized integrated error between simulation and experiments of STS transfers performed by different user groups.Box plots show the distribution of the errors with respect to the mean value (red lines), while dashed gray lines present the standard deviation (Color figure online)

Fig. 7
Fig. 7 Columns from left to right: simulation results of the human COM positions, joint positions, external assistive forces and moment, and joint torques during STS transfers for patients belong to the class of maximal assistance and having equal weakness in all joints (first row)

Fig. 8
Fig. 8 Assistive robot with actuated handles employed for the evaluation of proposed assistive STS transfers

Fig. 9
Fig. 9 An example of the achieved individualized robot handle trajectories (left handle position, right handle velocities) (Color figure online)

Fig. 11
Fig.11Comparison of simulation and experimental force trajectories for two elderly subjects able to fully stretch their body in the final standing configuration (top) and two elderly subjects not able to fully stretch their body in the final standing configuration (below).Solid and dotted lines are obtained measurements of the fourth and fifth STS transfer trials, while dashed lines are the expected force trajectories obtained by simulation (Color figure online)

Fig. 12
Fig. 12 Snapshots taken during the evaluation of the STS transfer assistance

First
T / t.Then, each iteration starts with an open loop control sequence τ k that is applied to the deterministic nonlinear and discretized forward dynamics xk+1 = xk + t f ( xk , τ k ) using standard Euler integration at sample k.Then, the dynamics and the cost function are quadratically approximated in the vicinity of the current trajectory.Both aforementioned approximations are expressed in terms of state and control deviations, i.e. δx k = x k − xk and δτ k = τ k − τ k , and are computed as follows,δx k+1 = (I + t f x )δx k + t ( f τ δτ k + δτ T k f τ x δx k ) + 0.5 t (δx T k f xx δx k + δτ T k f τ τ δτ k ) (10) c(δx, δτ ) = δx T k c x + δτ T k c τ + δτ T k c τ x δx k + 0.5(δx T k c xx δx k + δτ T k c τ τ δτ k ) (11)with func vars the partial derivative of function func with respect to variables ordered by vars and evaluated at ( xk , τ k ).
* k = l k + L k δx k with an open loop term (l k = −S −1 k R k ) and a feedback term (L k δx k = −S −1 k T k δx k ).Additional control constraints are taken into account by enforcing the open loop terms to lie inside of a constrained boundary.For each iteration i the approximate optimal control sequence τ (i+1) k is finally obtained by adding the newly calculated corrective control term and the control term of the last

Table 2
Mean value of the cost

Table 3
Anthropometric data of healthy subjects participating in the STS model validation