Conditional multiqubit gates are a key component for elaborate quantum algorithms. In a recent work, Rasmussen et al. [Phys. Rev. A 101, 022308 (2020)] proposed an efficient single-step method for a prototypical multiqubit gate, a Toffoli gate, based on a combination of Ising interactions between control qubits and an appropriate driving field on a target qubit. Trapped ions are a natural platform to implement this method, since Ising interactions mediated by phonons have been demonstrated in increasingly large ion crystals. However, the simultaneous application of these interactions and the driving field required for the gate results in undesired entanglement between the qubits and the motion of the ions, reducing the gate fidelity. In this work, we propose a solution based on adiabatic switching of these phonon-mediated Ising interactions. We study the effects of imperfect ground-state cooling and use spin-echo techniques to undo unwanted phase accumulation in the achievable fidelities. For gates coupling to all axial modes of a linear crystal, we calculate high-fidelity (>99%) N-qubit rotations with N=3–7 ions cooled to their ground state of motion and a gate time below 1 ms. Finally, we study the effect of laser intensity fluctuations and find that the proposed gate requires intensity stabilization with subpercentage noise levels. The high fidelities obtained also for large crystals could make the gate competitive with gate-decomposed, multistep variants of the N-qubit Toffoli gate, at the expense of requiring ground-state cooling of the ion crystal.