No doubt someone will give a better answer, and if this does not fit your requirements, feel free to downvote, but to my knowledge the PQ theory is successfully used in active power filters for both unbalanced and harmonic content conditions. The DQ theory may be more suited to VFDs give the control of the motors, but if it's about the calculation of powers, it can be done in 3 wire systems by using these equivalences (using Akagi's PQ theory):
$$\begin{align}
v_a&=v_{ab}-v_{ca} \tag{1a} \\
v_b&=v_{bc}-v_{ab} \tag{1b} \\
v_c&=v_{ca}-v_{bc} \tag{1c}
\end{align}$$
With these you can then proceed to calculate the \$\alpha\beta\$ components:
$$\begin{bmatrix}
v_{\alpha} \\
v_{\beta}
\end{bmatrix}
=\sqrt{\dfrac23}
\begin{bmatrix}
1 & -\dfrac12 & -\dfrac12 \\
0 & \dfrac{\sqrt3}{2} & -\dfrac{\sqrt3}{2}
\end{bmatrix}
\cdot
\begin{bmatrix}
v_a \\
v_b \\
v_c
\end{bmatrix} \tag{2}$$
The same for currents. The powers will then be:
$$\begin{bmatrix}
p \\
q
\end{bmatrix}
=
\begin{bmatrix}
v_{\alpha} & v_{\beta} \\
v_{\beta} & -v_{\alpha}
\end{bmatrix}
\cdot
\begin{bmatrix}
i_{\alpha} \\
i_{\beta}
\end{bmatrix} \tag{3}$$
The resulting powers will have both DC and oscillating values:
$$\begin{align}
p&=\bar{p}+\tilde{p} \tag{4a} \\
q&=\bar{q}+\tilde{q} \tag{4b}
\end{align}$$
As far as active power filters are concerned, there are many strategies, but for a more common shunt APF, constant power strategy, \$p\$ is filtered of its DC component and negated, $q$ is negated, and to these two there is an additional power called "loss power" coming from the DC bus regulation which is added:
$$\begin{bmatrix}
i_{C\alpha} \\
i_{C\beta}
\end{bmatrix}
=\dfrac{1}{v_{\alpha}^2+v_{\beta}^2}\cdot
\begin{bmatrix}
v_{\alpha} & v_{\beta} \\
v_{\beta} & -v_{\alpha}
\end{bmatrix}
\cdot
\begin{bmatrix}
-\tilde{p}+\bar{p}_{\text{loss}} \\
-q
\end{bmatrix} \tag{5}$$
Finally, the inverse Clarke is applied, and the currents are used for comepnsation:
$$\begin{bmatrix}
i_{Ca} \\
i_{Cb} \\
i_{Cc}
\end{bmatrix}
=\sqrt{\dfrac23}\cdot
\begin{bmatrix}
1 & 0 \\
-\dfrac12 & \dfrac{\sqrt3}{2} \\
-\dfrac12 & -\dfrac{\sqrt3}{2}
\end{bmatrix}
\begin{bmatrix}
i_{C\alpha} \\
i_{C_\beta}
\end{bmatrix} \tag{6}$$
A proof of concept can be simulated in SPICE. While I could show an exploded view with all the gory details, I'd say a schematic at block level can be read much better.
There are a few shortcuts I used, mostly for the sake of space, to be able to show al the schematic at once. The voltages are directly measured against a some common point, nul, but here they represent (1a,b,c), which still stand:
[EDIT] Note that the upper graph shows a grey label but an orange trace. That's the result of a .STEP, which has one of the traces (grey) as being null, and the other (orange), non-null. If you zoom in you'll see that the X axis is a slightly lighter shade of grey than the Y axis, when they should be the same -- that's the zero valued first trace. The exaplanation is at the end edit. [/EDIT]
Then, I used some emulated NTC, which is also deleted from the final schematic, but the currents will show a gradual rise. Lastly, the formulas above are for a constant power strategy, but I'm showing a sinusoidal current strategy, that's why you'll see a Clarke transform for the currents which is not used and, instead, some unity sine/cosine are used; the purpose was a better viewing of the graphs.
Ug is the grid (up, left), feeding two nonlinear loads: two bridges, one with a parallel RC (usual rectifier, 3 phase), and one with a series RL (but only on two phases). In the lower side there are the isolated voltage sensors [consider them (1a,b,c)], the Clarke matrices for both currents and voltages (2), calculation of powers (3) [with the modification mentioned above], selection of powers (ignore the i_zero, it's null), calculations of compensating currents in the \$\alpha\beta\$ domain (5), inverse Clarke (6), feeding the compensating currents back into the grid through the Ga, Gb, Gc current sources.
[EDIT 2] I forgot to post this last time, but the loads have changed to the regular bridge + R||C that you see, plus a simple series RL connected between phases a and b. Ultimately, it doesn't matter, the purpose was to have a nonlinear load and an imbalanced load, and to see how the PW theory works. [/EDIT 2]
Plotted are (top to bottom): powers calculated before and after the point of common coupling (PCC nodes), the output currents, the input currents, and the LL voltages showing unbalance (V(a) is 0.8 pu) and harmonics. At 96 ms the APF turns on, showing clean sinusoidal currents on the input side. If the power strategy would have been used, the currents would have still had harmonics (though greatly reduced), while the input power would have been flat:
As per the comments below, (1) might not be intuitive, so here's why it holds (there's a typo in the 2nd comment, 2 instead of 3):
$$\begin{align}
\sin(x+\theta)&=\cos(\theta)\sin(x)+\sin(\theta)\cos(x) \tag{7a} \\
\sin(x-\theta)&=\cos(\theta)\sin(x)-\sin(\theta)\cos(x) \tag{7b} \\
(7a)+(7b)&=2\cos(\theta)\sin(x) \tag{7c} \\
v_{ab}&=v_a-v_b \\
v_{ab}-v_{ca}&=v_a-v_b-(v_c-v_a) \\
{}&=v_a-v_b-v_c+v_a \\
{}&=2v_a-v_b-v_c \tag{8} \\
v_a&=\sin(x),\;v_b=\sin(x-\dfrac23\pi),\;v_c=\sin(x+\dfrac23\pi) \\
(8)&=2\sin(x)-2\cos(\theta)\sin(x)\Biggr|_{\theta=\frac23\pi} \\
{}&=2\sin(x)-2\cdot\left(-\dfrac12\right)\sin(x) \\
{}&=3\sin(x) \tag{9}
\end{align}$$
These relations hold only as long as there is no imbalance. But -- and here is the key -- the Clarke transform cancels the zero component, which is not needed in a 3-wire system. This is why you can see a non-zero trace in the upper plot of the 1st picture (orange), versus a zero valued one for the other (grey, overlayed on the X axis).
To further prove this, in the previous pictures the generator is disposed in Y, even if the common (the NUL pin) is clearly connected to a capacitor in parallel with a 1 MΩ resistor (and a series 1 Ω to limit the potential currents) -- it is floating. So, this time, here is a Δ generator, and how the waveforms look like before and after the Clarke matrix, for each (note the imbalance in V4):
The bottom plot shows the LN voltages, referenced to the SPICE ground (ugly, meaning the voltage across C7 will not be zero), the middle shows the derived voltages, and at the top there are the resulting currents for a constant power strategy -- showing that the scheme works. I've left the αβ components separate, for better viewing:
Here, too, the zero component is non-null, but this time it's on the abc side, since they are derived from Δ, not Y.
So, as a conclusion, it doesn't matter how the derived abc come out, the Clarke matrix and the subsequent power calculations will correctly account for the 3- or 4-wire system, since the purpose of the Clarke matrix is to consider the three-phase as a whole, not as individual, summed components.
