**From:** Giacomo Fiorin (*giacomo.fiorin_at_gmail.com*)

**Date:** Tue Jul 22 2014 - 17:31:23 CDT

**Next message:**Jo, Sunhwan: "Re: Cost of a colvar calculation"**Previous message:**Jason Swails: "Re: NAMD_energy_plot"**In reply to:**Jo, Sunhwan: "Re: Cost of a colvar calculation"**Next in thread:**Jo, Sunhwan: "Re: Cost of a colvar calculation"**Reply:**Jo, Sunhwan: "Re: Cost of a colvar calculation"**Messages sorted by:**[ date ] [ thread ] [ subject ] [ author ] [ attachment ]

Hello Sunhwan, you can also use the option enableFitGradients to achieve

the same result.

Regards

Giacomo

On Tue, Jul 22, 2014 at 2:06 PM, Jo, Sunhwan <sjo_at_anl.gov> wrote:

*> RMSD colvar is particularly problematic if you use large number of CPUs. I
*

*> also had about 10x slow down in performance when I had to include RMSD
*

*> colvar in BG/Q. In addition, the scaling performance saturate pretty
*

*> quickly. At the time, I had ~160K atoms and the scaling saturated after 32
*

*> cores with colvar turned on. I had total of 9 clovers (4 RMSDs).
*

*>
*

*> At the time, I have found a bottleneck in the colvar implementation and
*

*> I’d like to share it here. Below is the modification that I made in
*

*> colvaratoms.C file and it almost speed up the simulation about 9x. The
*

*> scaling also improved up to 128 cores. Find the lines ends with the comment
*

*> “MOD”.
*

*>
*

*> The bottleneck was gradient calculation of the best fit rotation matrix of
*

*> the reference atoms. Knowing CHARMM skips gradient of rotation matrix, so I
*

*> decided to skip it. It is possible this subroutine is required for ABF but
*

*> I wasn’t using it so I didn’t check. I also didn’t have RMSD colvar slowing
*

*> the simulation in other cluster (i.e., Kraken) where I only had to use 1 or
*

*> 2 nodes.
*

*>
*

*>
*

*> > void cvm::atom_group::calc_fit_gradients()
*

*> > {
*

*> > if (b_dummy) return;
*

*> >
*

*> > if ((!b_center) && (!b_rotate)) return; // no fit
*

*> >
*

*> > if (cvm::debug())
*

*> > cvm::log ("Calculating fit gradients.\n");
*

*> >
*

*> > atom_group *group_for_fit = ref_pos_group ? ref_pos_group : this;
*

*> > group_for_fit->fit_gradients.assign (group_for_fit->size(),
*

*> cvm::rvector (0.0, 0.0, 0.0));
*

*> >
*

*> > if (0 && b_center) { // MOD
*

*> > // add the center of geometry contribution to the gradients
*

*> > for (size_t i = 0; i < this->size(); i++) {
*

*> > // need to bring the gradients in original frame first
*

*> > cvm::rvector const atom_grad = b_rotate ?
*

*> > (rot.inverse()).rotate ((*this)[i].grad) :
*

*> > (*this)[i].grad;
*

*> > for (size_t j = 0; j < group_for_fit->size(); j++) {
*

*> > group_for_fit->fit_gradients[j] +=
*

*> > (-1.0)/(cvm::real (group_for_fit->size())) *
*

*> > atom_grad;
*

*> > }
*

*> > }
*

*> > }
*

*> >
*

*> > if (0 && b_rotate) { // MOD
*

*> >
*

*> > // add the rotation matrix contribution to the gradients
*

*> > cvm::rotation const rot_inv = rot.inverse();
*

*> > cvm::atom_pos const cog = this->center_of_geometry();
*

*> >
*

*> > for (size_t i = 0; i < this->size(); i++) {
*

*> >
*

*> > cvm::atom_pos const pos_orig = rot_inv.rotate ((b_center ?
*

*> ((*this)[i].pos - cog) : ((*this)[i].pos)));
*

*> >
*

*> > for (size_t j = 0; j < group_for_fit->size(); j++) {
*

*> > // calculate \partial(R(q) \vec{x}_i)/\partial q) \cdot
*

*> \partial\xi/\partial\vec{x}_i
*

*> > cvm::quaternion const dxdq =
*

*> > rot.q.position_derivative_inner (pos_orig, (*this)[i].grad);
*

*> > // multiply by \cdot {\partial q}/\partial\vec{x}_j and add it
*

*> to the fit gradients
*

*> > for (size_t iq = 0; iq < 4; iq++) {
*

*> > group_for_fit->fit_gradients[j] += dxdq[iq] * rot.dQ0_1[j][iq];
*

*> > }
*

*> > }
*

*> > }
*

*> > }
*

*> > if (cvm::debug())
*

*> > cvm::log ("Done calculating fit gradients.\n");
*

*> > }
*

*>
*

*>
*

*>
*

*> Thanks,
*

*> Sunhwan
*

*>
*

*>
*

*> On Jul 22, 2014, at 6:27 AM, George Patargias <gpat_at_bioacademy.gr> wrote:
*

*>
*

*> > Hello,
*

*> >
*

*> > I am comparing the Benchmark times for a given system (~258,000 atoms)
*

*> using
*

*> > NAMD_CVS-2014-06-02_Source version with and without the colvar module on.
*

*> >
*

*> > When the colvar module is *off* (equilibrium MD), I get something like
*

*> >
*

*> > Benchmark time: 256 CPUs 0.0268481 s/step 0.155371 days/ns 540.59 MB
*

*> memory
*

*> >
*

*> > When the colvar module is *on* (moving harmonic restraint on an RMSD
*

*> > colvar calculated from 1981 C-alpha atoms), I get
*

*> >
*

*> > Benchmark time: 256 CPUs 0.320515 s/step 1.85483 days/ns 569.555 MB
*

*> memory
*

*> >
*

*> > which is 11-12 times slower.
*

*> >
*

*> > Is this the actual computational cost of a colvar calculation?
*

*> >
*

*> > Also the scaling of this particular colvar calculation is not good:
*

*> >
*

*> > Benchmark time: 48 CPUs 0.331787 s/step 1.92006 days/ns
*

*> > Benchmark time: 128 CPUs 0.322472 s/step 1.86616 days/ns
*

*> > Benchmark time: 256 CPUs 0.320515 s/step 1.85483 days/ns
*

*> >
*

*> > Thanks in advance!
*

*> >
*

*> > George
*

*> >
*

*> > Dr. George Patargias
*

*> > Postdoctoral Research Fellow
*

*> > Biomedical Research Foundation
*

*> > Academy of Athens
*

*> > 4, Soranou Ephessiou
*

*> > 115 27
*

*> > Athens
*

*> > Greece
*

*> >
*

*> > Office: +302106597568
*

*> >
*

*>
*

*>
*

**Next message:**Jo, Sunhwan: "Re: Cost of a colvar calculation"**Previous message:**Jason Swails: "Re: NAMD_energy_plot"**In reply to:**Jo, Sunhwan: "Re: Cost of a colvar calculation"**Next in thread:**Jo, Sunhwan: "Re: Cost of a colvar calculation"**Reply:**Jo, Sunhwan: "Re: Cost of a colvar calculation"**Messages sorted by:**[ date ] [ thread ] [ subject ] [ author ] [ attachment ]

*
This archive was generated by hypermail 2.1.6
: Wed Dec 31 2014 - 23:22:39 CST
*