how to correctly execute su from Android application

This way:
Runtime.getRuntime().exec(new String[]{"/system/bin/su", "-c", "setprop ctl.stop zygote"});
If you are getting failures like this:
W/su ( 1043): request rejected (0:0->0:0 /system/bin/setprop)
that means that

• you didn't send "-c", or
• you didn't give it a parameter, or
  
• you gave it more than one parameter

You have to make sure that after "-c" you only send one parameter. Alternate syntax would be this:
Runtime.getRuntime().exec("/system/bin/su -c 'setprop ctl.stop zygote'");