An Unknown Interference Suppression Scheme for Advanced Antenna Systems

L. T. Trang${}^{\mathbf{\text{1,2}}}$, N. V. Cuong${}^{\mathbf{\text{1}}}$, H. T. P. Thao${}^{\mathbf{\text{2}}}$, and T. V. Luyen${}^{\mathbf{\text{1}}}$

¹School of Electrical and Electronic Engineering
Hanoi University of Industry, Hanoi, Vietnam
tranglt@haui.edu.vn, cuongnv@haui.edu.vn, luyentv@haui.edu.vn

²Faculty of Electronics and Telecommunications
Electric Power University, Hanoi, Vietnam
thaohp@epu.edu.vn

Submitted On: December 4, 2024; Accepted On: February 27, 2025

ABSTRACT

An unknown interference suppression scheme for advanced antenna systems has been proposed to address critical challenges in enhancing wireless communication networks. This scheme focuses on improving beamforming capabilities and spectral efficiency while minimizing the impact of unknown interference. The ability to suppress unknown interference is achieved through a fitness function that does not rely on prior knowledge of interference characteristics. This function is designed based on the assumption that the desired signal is received through the main lobe, while interference predominantly resides in the sidelobes. By incorporating a constraint handling technique, specifically the static penalty method, the fitness function ensures that total output power is minimized only when interference power in the sidelobes is effectively reduced. Additionally, the optimization process is streamlined by reducing the number of optimization variables, focusing on uniform rectangular arrays with square element distributions. Metaheuristic algorithms, including the Binary Bat Algorithm, Binary Grey Wolf Optimization, and Binary Whale Optimization Algorithm, are applied to adaptively suppress unknown interference while reducing computational complexity. The proposed scheme significantly enhances advanced antenna systems performance by steering adaptive nulls toward unknown interference sources, ensuring robustness in dynamic wireless environments.

Index Terms: Binary Grey Wolf Optimization, constraint handling techniques, static penalty method, uniform rectangular arrays, unknown interference suppression.

I. INTRODUCTION

Advanced antenna systems (AAS) are a key component of modern wireless communication networks, including 5th Generation (5G), beyond 5G, and emerging 6G technologies. These systems significantly enhance capacity, coverage, and energy efficiency. A key feature of AAS is their flexible beam control, enabled by technologies such as massive MIMO, beamforming, and adaptive antenna arrays [1]. These capabilities allow for more efficient spectrum utilization, higher throughput, and improved connectivity, all while minimizing interference. In the context of next-generation networks, AAS play a critical role in supporting ultra-reliable low-latency communications, massive machine-type communications, and enhanced mobile broadband, which are essential for deploying 5G, beyond 5G, and 6G [2].

One of the most important capabilities of AAS is their ability to precisely control beam patterns, enabling targeted signal transmission and reception. This capability enhances link quality, expands coverage, and supports higher user densities, especially in urban settings or industrial IoT applications. Additionally, AAS can dynamically adapt to changing network conditions, optimizing radiation patterns in real time to avoid interference and maximize system performance. However, as the number of antennas in these systems increases, so too does the number of optimization variables, complicating the optimization process and making it more time consuming. This presents a significant challenge, especially in real-time processing for 5G, beyond 5G, and 6G networks. Additionally, AAS continues to face substantial challenges in managing unknown interference from external sources.

In practical wireless communication environments, interference can arise from unintended sources such as co-channel transmissions, multipath effects, and external electronic devices. This interference is often unpredictable and lacks prior statistical or spatial information, making it “unknown interference”. This interference is often unpredictable in both direction and intensity, leading to system degradation, reduced data rates, and increased latency. Traditional suppression methods assume prior knowledge of interference characteristics, limiting their effectiveness in real-world scenarios where unknown interference is common. As wireless environments grow more complex with the development of 5G, beyond 5G, and 6G, developing robust schemes for managing unknown interference is becoming increasingly critical [2, 3].

Interference suppression is a critical challenge in wireless communication, particularly in scenarios where interference characteristics are unknown. Various methods have been explored in the literature to address this issue. Known interference is typically addressed through methods like beamforming or adaptive filtering, both of which require prior knowledge of the interference source [4, 5]. For unknown interference, traditional techniques, such as blind source separation [6], have been widely used for signal extraction. Deep learning-based approaches have also gained significant attention for their ability to learn interference patterns directly from data, as discussed in [7, 8].

Recent advancements in interference suppression further extend these approaches. For instance, SlickScatter [9] introduces an interference-insensitive WiFi backscatter system that retrieves backscatter signals despite unknown ambient interference. Similarly, automatic modulation recognition techniques [10] leverage graph neural networks to classify unknown interference signals. Additionally, interference source positioning methods based on near-field scanning [11] provide effective localization of interference sources. In the context of hardware-based solutions, a 2-D MIMO receiver array [12] demonstrates autonomous spatial filtering to suppress unknown interference. Bayesian methods [13] have also been proposed to characterize unknown interference power in wireless networks, offering insights for rate adaptation. Furthermore, adaptive beamforming techniques [14] enhance robustness by iteratively placing radiation pattern nulls to suppress sidelobes and mitigate unexpected interference.

A promising scheme involves combining metaheuristic algorithms with constrained handling techniques (CHTs). Metaheuristic algorithms like Binary Bat Algorithm (BBA), Binary Grey Wolf Optimization (BGWO), and Whale Optimization Algorithm (BWOA) have shown promise in optimizing beamforming parameters, particularly in the presence of unknown interference. BBA, inspired by the echolocation behavior of bats, is known for its rapid exploration of the scheme space, offering quick convergence during early optimization stages [15, 16]. However, it can struggle to balance exploration and exploitation in more complex environments [17]. BWOA, which mimics the hunting strategies of humpback whales, excels in fine-tuning schemes and thoroughly exploring the search space, although it may take longer to fully converge [18]. BGWO, modeled after the social behavior of grey wolves during hunting, offers a balanced approach to exploration and exploitation, making it well-suited for complex, multi-dimensional optimization problems [19]. By leveraging the strengths of BBA, BGWO, and BWOA, combined with CHTs, a scheme can be achieved for beamforming, minimizing sidelobe levels (SLL), placing nulls in interference directions, and optimal resource allocation in the presence of both known and unknown interference [20].

This paper proposes a scheme for suppressing unknown interference in AAS without requiring prior knowledge of interference characteristics. The scheme achieves this by constructing a fitness function that minimizes total output power while ensuring that desired signals remain unaffected. This function is formulated using a constraint handling technique, specifically the static penalty method (SPM), which effectively suppresses interference residing in the sidelobes. To enhance computational efficiency, the scheme integrates metaheuristic algorithms with CHTs, optimizing interference suppression while minimizing the number of optimization variables in uniform rectangular arrays (URA). This reduces search time and complexity, making the approach particularly suitable for advanced communication networks, including 5G, beyond 5G, and emerging 6G technologies.

II. CONSTRAINED HANDLING TECHNIQUES AND CONSTRAINED OPTIMIZATION PROBLEMS

A. Constrained handling techniques

CHTs are a set of methods designed to address optimization problems where constraints must be satisfied while optimizing an objective function. CHTs play a critical role in ensuring that schemes meet predefined constraints, making them indispensable in solving constrained optimization problems. Their primary advantage lies in their ability to guide the optimization process towards feasible regions of the search space, avoiding schemes that violate the constraints. By balancing the search space exploration with the adherence to constraints, CHTs enhance the reliability and practicality of optimization outcomes, particularly in complex, multi-dimensional problem domains. The significance of CHTs is further amplified when dealing with constrained optimization problems, where schemes must not only maximize or minimize an objective function but also stay within the defined boundaries of the problem. These techniques are particularly useful in non-convex or discontinuous scheme spaces, where finding feasible schemes is challenging. CHTs encompass a range of approaches, from simple penalty methods, which impose costs for constraint violations, to more sophisticated techniques, like stochastic ranking and adaptive penalty methods, that dynamically adjust their behavior to meet the problem’s constraints.

CHTs become even more crucial when applied to AAS due to the unique challenges these systems face. AAS requires precise control over beamforming patterns and interference mitigation while adhering to strict physical and operational constraints, such as maintaining the desired beam shape, minimizing SLLs, and steering nulls towards interference sources. These requirements naturally lead to constrained optimization problems where the goal is to optimize beamforming parameters without violating the system’s constraints.

To address these challenges in AAS, four of the most commonly used CHTs in the literature are: (i) penalty methods, (ii) feasibility rules, (iii) the $\epsilon$-constrained method, and (iv) stochastic ranking. Each of these techniques offers a unique way of handling constraints during the optimization process. Among these, objective function penalization stands out as the most traditional and widely employed approach. This method transforms a constrained problem into an unconstrained one by adding penalty terms to the objective function. The idea is to penalize schemes that violate constraints, making the optimization algorithm less likely to explore infeasible regions. Over time, various penalty-based methods have evolved, including static, dynamic, adaptive, and death penalty techniques, each tailored to handle different levels of problem complexity and optimization goals [21].

In summary, CHTs provide a powerful framework for solving the constrained optimization problems encountered in AAS, ensuring that both performance and constraint compliance are achieved. This makes them essential for the effective design and operation of advanced wireless communication networks, especially as these networks evolve into 5G, 5G beyond, and 6G systems.

B. Constrained optimization problems

A constrained optimization problem typically involves finding an optimal scheme to an objective function subject to a set of constraints. The general form of a constrained optimization problem can be expressed as [22]:

$\mathrm{minimize}$	$f(\mathbf{\text{x}}):$
		$g_j\mathbf{\text{(x)}}\le 0,j=1,2,\mathrm{\dots },j$
$\mathrm{subject}\mathrm{to}$	$h_k\mathbf{\text{(x)}}=0,k=1,2,\mathrm{\dots },K,$	(1)
		$x_i^l\le x_i\le x_i^u,i=1,2,\mathrm{\dots },n$

where:

$f(\mathbf{\text{x}}):$ objective function concerning the vector variable $\mathbf{x}$

$g_j(\mathbf{\text{x}}):$ inequality constraints

$h_k(\mathbf{\text{x}}):$ equality constraints

$x_i^l,$$x_i^u$ represents the lower and upper limit values, respectively, of component $x_i$ in $\mathbf{x}$.

In this article, SPM is applied, and then the constrained optimization problem (B.) is transformed into an unconstrained optimization problem (2).

In the static penalty approach outlined by [23], the penalty coefficient escalates as the level of violation increases, as noted in [24]. Although the penalty functions remain unchanged, a static penalty function is proposed that adapts the static penalty parameter based on the severity of violations, as discussed in [25]. In the context of the static penalty function presented in [26], a constrained problem defined in (B.) is converted into a non-constrained form:

$$Minimize\left\{f(\mathbf{\text{x}})+\xi v(\mathbf{\text{x}})\right\},$$

(2)

where $\xi v(\mathbf{\text{x}})$ presents a penalty term.

Inspired by the optimization in (2), the fitness function of the optimization in this study is defined as:

$$F(\mathbf{x},\xi )=f(\mathbf{x})+\xi v(\mathbf{\text{x}}).$$

(3)

In this context, each unmet constraint affects $\mathbf{x}$ by imposing a penalty equal. These penalties are aggregated and multiplied by $\xi ,$ the penalty parameter, which is then balanced against $f(\mathbf{\text{x}})$. Therefore, if the magnitude of the penalty term is minor compared to that of $f(\mathbf{\text{x}})$, it is highly likely that minimizing $F(\mathbf{\text{x}},\xi )$ will not yield a feasible $\mathbf{x}$ for the original problem. Conversely, if $\xi$ is sufficiently large, the penalty for any constraint violation will be substantial enough that minimizing the fitness function will lead to a feasible scheme.

Metaheuristic algorithms, such as BBA, BGWO, and BWOA, are particularly well-suited for solving complex constrained optimization problems. These algorithms efficiently explore the scheme space, making them potential candidates for optimizing AAS where multiple constraints need to be satisfied [22].

CHT transforms constrained optimization problems into single-objective optimization problems. By integrating penalty terms into the fitness function, metaheuristic algorithms can focus on optimizing a single criterion, such as interference suppression, while ensuring that the constraints are satisfied. This approach is particularly useful in AAS, where optimizing beamforming requires balancing multiple conflicting objectives.

III. PROBLEM FORMULATION

This paper examines URA, a type of uniform planar array of half-wavelength dipoles as shown in Fig. 1. The antenna array pattern can be represented as:

$P(\theta ,\varphi )$	$=EF(\theta ,\varphi )AF(\theta ,\varphi )$
		$=EF(\theta ,\varphi ){\displaystyle \sum _{m=0}^{M-1}}{\displaystyle \sum _{n=0}^{N-1}}{w}_{m,n}{e}^{j\left(m{\psi }_z+n{\psi }_y\right)},$	(4)

where:

$EF$ and $AF$ are the element factor and the array factor of the dipole $(\theta ,\varphi )$, respectively.

Figure 1: Uniform rectangular array with $M\times N$elements.

$EF(\theta ,\varphi )=\frac{\cos \left(\frac{\pi }{2}\cos \left(\theta \right)\right)}{\sin \left(\theta \right)}$,

${\psi }_z=\kappa d_z\cos \left(\theta \right);$${\psi }_y=\kappa d_y\sin \left(\theta \right)\sin \left(\varphi \right);$$\kappa =2\pi /\lambda ;$

$w_{m,n}=a_{m,n}{e}^{j{\delta }_{m,n}}$ is the complex weight of the ${(m,n)}^{th}$ element, where $a_{m,n}$ and ${\delta }_{m,n}$ are the amplitude and the phase, respectively.

The main lobe can be steered towards the direction $({\theta }_0,{\varphi }_0)$ by setting the phase shift of the ${(m,n)}^{th}$ antenna element as:

$${\delta }_{m,n}=-\kappa \left(m{d}_z\cos \left({\theta }_0\right)+n{d}_y\sin \left({\theta }_0\right)\sin \left({\varphi }_0\right)\right).$$

(5)

The array pattern for URAs can be expressed in the matrix form as:

$$P(\theta ,\varphi )=EF(\theta ,\varphi )\mathbf{s}(\theta ,\varphi )\mathbf{w},$$

(6)

where:

steering vector is expressed as:

$$\mathbf{\text{s}}(\theta ,\varphi )=[{\mathrm{e}}^{\mathrm{j}\left({\mathrm{m}}_0{\psi }_{\mathrm{z}}+{\mathrm{n}}_0{\psi }_{\mathrm{y}}\right)},\mathrm{\dots },{\mathrm{e}}^{\mathrm{j}\left({\mathrm{m}}_{\mathrm{M}-1}{\psi }_{\mathrm{z}}+{\mathrm{n}}_{\mathrm{N}-1}{\psi }_{\mathrm{y}}\right)}],$$

(7)

optimal weight vector is expressed as:

$$\mathbf{\text{w}}={[w_{0,0},\mathrm{\dots },w_{M-1,N-1}]}^T.$$

(8)

To achieve the desired array pattern with $K$ nulls in the interference directions $({\theta }_k,{\varphi }_k),$ the problem with respect to w can be formulated as (variable w corresponds to the variable $\mathbf{x}$ in section II):

$\underset{w}{max}\mathit{\hspace{1em}\hspace{1em}}P(\mathbf{\text{w}},{\theta }_0,{\varphi }_0)$
$\mathrm{s}.\mathrm{t}.\mathit{\hspace{1em}\hspace{1em}\hspace{0.25em}}P(\mathbf{\text{w}},{\theta }_k,{\varphi }_k)=0\mathit{\hspace{1em}\hspace{1em}}\forall k=1,\mathrm{\dots },K.$	(9)

IV. PROPOSED SCHEME

In this study, we propose an unknown interference suppression scheme in AAS, specifically for URAs. The scheme focuses on two main improvements. First, we reduce the number of optimization variables, which shortens the computational time, an important factor in optimizing the efficiency of AAS. Second, we apply CHT, specifically the SPM, to enforce constraints on beam pattern shaping and null placement in interference directions. This helps simplify the optimization process, thereby improving the overall performanceof AAS.

Improvement 1: Minimize the number of variables that need to be optimized (consider the array containing elements distributed in a square). This improvement focuses on reducing the number of variables that need to be optimized, specifically in arrays with square element distributions. The goal is to simplify the optimization process, which is crucial for improving computational efficiency in AAS.

---

Algorithm 1: The algorithm determines the optimal weights while reducing the number of phase variables in the optimization search.

---

1: Input:

2: Number of antennas in z-axis $M$ and y-axis $N.$ The array contains elements arranged in a square configuration, so $M=N$

3: Binary schemes ${\mathbf{\text{s}}}_{\mathrm{bin}}$ from metaheuristics algorithms

4: Resolution of phase shifters $N_{bits}$

5: Operation:

6: Determine the binary scheme of an individual in metaheuristic algorithms. The number of binary bits in the $ith$ variable, $s_{\mathrm{bin},i}$ with $i=1,\mathrm{\dots },M,$ represents the number of phase shifts for an $N_{bits}$ phase shifter

$${\mathbf{\text{s}}}_{\mathrm{bin}}={[s_{\mathrm{bin},1},\mathrm{\dots },s_{\mathrm{bin},M}]}^T.$$

(10)

7: Convert the binary scheme to a real number. In this paper, the number of binary bits in the variable $s_{\mathrm{bin},i}$ is set to 1, and $N_{bits}$ is set to 5:

$${\mathbf{\text{s}}}_{\mathrm{o}}={[s_1,\mathrm{\dots },s_M]}^T=\frac{\mathrm{bin2dec}\left({\mathbf{\text{s}}}_{\mathrm{bin}}\right)}{2^{N_{bits}}-1}.$$

(11)

8: Determine the phase of elements on the z-axis and y-axis based on odd symmetry characteristics:

9: If $M$ and $N$ are even, then:

$$\begin{array}{c}{\mathbf{\delta }}_y={[s_1,\mathrm{\dots },s_{M\mathrm{/}2},-s_{M\mathrm{/}2},\mathrm{\dots },-s_1]}^T\hfill \\ {\mathbf{\delta }}_z={[s_{M\mathrm{/}2+1},\mathrm{\dots },s_M,-s_M,\mathrm{\dots },-s_{M\mathrm{/}2+1}]}^T\hfill \end{array}.$$

(12)

10: If M and N are odd, then:

$$\begin{array}{c}{\mathbf{\delta }}_y={[s_1,\mathrm{\dots },s_{M\mathrm{/}2},0,-s_{M\mathrm{/}2},\mathrm{\dots },-s_1]}^T\hfill \\ {\mathbf{\delta }}_z={[s_{M\mathrm{/}2+1},\mathrm{\dots },s_M,0,-s_M,\mathrm{\dots },-s_{M\mathrm{/}2+1}]}^T\hfill \end{array}.$$

(13)

11: Perform the outer sum operation to determine the phase for all elements in the URAs:

$${\mathbf{\delta }}_{yz}=\mathrm{vec}\left({\mathbf{\delta }}_y\oplus {\mathbf{\delta }}_z\right),$$

(14)

where:

$\oplus$ the outer sum combines row and column positions to calculate the phase for each antenna in the URA.

12: The optimal phase based on the reference phase, for example, the phase of the weights when steering the main beam, is determined as:

$${\mathbf{\delta }}_{\mathrm{o}}={\mathbf{\delta }}_{yz}+{\mathbf{\delta }}_{\mathrm{ref}}.$$

(15)

13: Optimal weights when reducing the number of phase variables to be searched:

$$\begin{array}{c}{\mathbf{\text{w}}}_{\mathrm{o}}={\mathbf{\text{a}}}_{\mathrm{ref}}\odot e^{j{\mathbf{\delta }}_{\mathrm{ref}}}\odot e^{j{\mathbf{\delta }}_{yz}}\hfill \\ ={\mathbf{\text{w}}}_{\mathrm{ref}}\odot e^{j{\mathbf{\delta }}_{yz}}={\mathbf{\text{a}}}_{\mathrm{ref}}\odot e^{j{\delta }_{\mathrm{o}}}\hfill \end{array},$$

(16)

where:

${\mathbf{\text{w}}}_{\mathrm{o}}$ is optimal weight

${\mathbf{\text{a}}}_{\mathrm{ref}}$ is the array response vector corresponding to a specific reference direction. It represents the antenna array’s response to a signal arriving from that direction.

$\odot$ is the Hadamard product.

14: Output:

15: Optimal weight coefficients: ${\mathbf{\text{w}}}_{\mathrm{o}}.$

---

Improvement 2: Using CHT, specifically the SPM, to construct the fitness function in the optimization process.

If the main lobe is steered toward $({\varphi }_0,{\theta }_0),$ most of the high-magnitude sidelobes occur at $(\varphi =[-180{}^{\circ }:180{}^{\circ }],\theta ={\theta }_0)$ and $(\varphi ={\varphi }_0,\theta =[0{}^{\circ }:$ $180{}^{\circ }]).$ Nulls can be positioned freely in 3D space, but interferences that occur in high-power sidelobes are the most problematic. Therefore, this paper assumes that interferences arise at $(\varphi =[-90{}^{\circ }:90{}^{\circ }],\theta ={\theta }_0).$

Therefore, the term $f(\mathbf{\text{w}})$ utilized for maintaining the desired main lobe is represented as:

$f(\mathbf{\text{w}})$

$={\displaystyle \sum _{\varphi ={\varphi }_0-{\varphi }_{\mathrm{FNBW}}\mathrm{/}2}^{{\varphi }_0+{\varphi }_{\mathrm{FNBW}}\mathrm{/}2}}{\left|P_{\mathrm{o}}(\mathbf{\text{w}},{\theta }_0,\varphi )-P_{\mathrm{ref}}(\mathbf{\text{w}},{\theta }_0,\varphi )\right|}^2$

where:

• $P_{\mathrm{ref}}(\mathbf{\text{w}},{\theta }_0,\varphi )$ and $P_{\mathrm{o}}(\mathbf{\text{w}},{\theta }_0,\varphi )$ correspond to the reference pattern and the optimized patterns using metaheuristic algorithms, respectively.

• ${\theta }_{\mathrm{FNBW}}$ and ${\varphi }_{\mathrm{FNBW}}$ correspond to the elevation and azimuth angles at the first null beamwidth (FNBW).

Generally, both the desired signal and interference sources reach the receiving arrays simultaneously. The desired signal is assumed to enter through the main lobe, whereas interference is distributed across the sidelobes. To effectively suppress unknown interference without requiring prior knowledge, we design the fitness function based on total output power minimization. This function incorporates a CHT, specifically the SPM, which ensures that only interference power is minimized while maintaining the main lobe for the desired signal. The term $v(\mathbf{\text{w}})$ is formulated to impose nulls in interference directions by leveraging the total received power across all antenna elements. Since the interference resides in the sidelobes, the algorithm adaptively adjusts weights to minimize interference impact while preserving signal integrity. This enables robust suppression of unknown interference, making the scheme effective even in scenarios with unpredictable interference sources.

The term $v(\mathbf{\text{w}})$, used to impose nulls in the directions of interference, is defined by the total output power of all received signals including both the desired signal and interference. The output power is computed by summing the product of signal weights and the magnitude of the array pattern in the directions of the desired $({\theta }_0,{\varphi }_0)$and interfering signals [27]:

$$v\left(\mathbf{\text{w}}\right)=\frac{1}{\sum \left|\mathbf{\text{w}}\right|}\sum _{k=1}^K{\left|si{g}_k{P}_{\mathrm{o}}(\mathbf{\text{w}},{\theta }_k,{\varphi }_k)\right|}^2,$$

(18)

where:

• $si{g}_k$: signal strength,

• $P_{\mathrm{o}}(\mathbf{\text{w}},{\theta }_k,{\varphi }_k)$ is the pattern for $k^{th}$ signal,

• $K$ is the total number of incoming signals.

Thus, the fitness function for addressing the unconstrained problem can be expressed as:

$$F(\mathbf{\text{w}},\xi )=\frac{f\left(\mathbf{\text{w}}\right)}{\xi }+v(\mathbf{\text{w}}).$$

(19)

The optimization algorithm used to illustrate the proposed scheme is based on the BGWO algorithm. The flowchart of the proposed scheme to improve unknown interference suppression in AAS, particularly for URAs, is shown in Fig. 2. This scheme combines two key improvements: minimizing optimization variables to reduce computational complexity and integrating CHT, specifically SPM, to improve interference rejection. The stopping criteria for the BGWO algorithm are defined by reaching the maximum number of iterations.

Figure 2: Flowchart of the proposed scheme to improve unknown interference suppression.

Below is a description of how to improve unknown interference suppression in AAS, particularly for URAs.

Initialize:

• The initial setup involves defining input data such as the number of array elements, the population size, the penalty factor, assumed direction of arrival of interferences, stopping condition (or maximum number of iterations), and the radiation pattern of the array elements.

• Define the objective function $f(\mathbf{\text{w}})$ from (IV.) and the term $v(\mathbf{\text{w}})$, used to impose nulls in the directions of interference, from (17), in which the array pattern is computed according to (4.).

• Mapping solutions (sets of weights) to locations of wolves in the population during the optimization process.

Find the weight vector with reduced phase variables:

After updating the positions of the Alpha, Beta, and Delta wolves, Algorithm 1 is implemented as described in Improvement 1.

Calculate fitness function by CHTs:

The fitness function, utilizing the SPM, is detailed in Improvement 2.

Find the best solution based on BGWO:

The beamformer iteratively computes and explores the current optimal solution using the BGWO. The process persists until the termination criterion is satisfied. Subsequently, the final optimal solution is acquired.

Construct of array element weights:

The beamformer establishes the corresponding weights for each URAs element based on the optimized solution. Pattern nulling is performed using these weights.

V. NUMERICAL RESULTS AND DISCUSSION

To assess the effectiveness of the proposed unknown interference suppression scheme, three metaheuristic algorithms were employed: BBA, BGWO, and BWOA. Among these, BGWO was selected as the primary optimization algorithm due to its well-balanced exploration and exploitation capabilities in complex search spaces. The simulations were conducted on three distinct scenarios to evaluate the unknown interference suppression performance of the proposed scheme for AAS.

The parameter setup for the simulations is as follows. The population size is set to $50$, with a maximum of $30$ iterations. The URA has dimensions of $M=N=11$, with SLL constraint of $-25$ dB. The angle step size is 0.5 degrees. Additionally, the number of phase bits for optimization is set to 5, and the penalty factor is 10. The signal-to-interference ratio of the k-th interference is $-30$ dB. The results across all scenarios represent the average values of $100$ independent simulations. These simulation results are detailed in the following sections, where each scenario is discussed along with the corresponding parameters and findings.

A. Effect of penalty parameter

In this scenario, we evaluate the impact of the penalty parameter on the performance of unknown interference suppression. The objective is to assess how different penalty values affect the suppression of interference at the desired angle $\theta ={25}^{\circ }$, while also considering related factors such as null depth level (NDL) and SLL. Simulation results with the BGWO algorithm show:

Figure 3: The plot illustrates the NDL at 25◦ as a function of the penalty parameter ξ for four signalto- interference ratios (SIR): 0 dB, −10 dB, −20 dB, and −30 dB using the BGWO algorithm. At 0 dB SIR, the NDL stabilizes around −20 dB when the penalty ξ reaches approximately 1e4, indicating consistent nulling performance with minimal sensitivity to the penalty parameter. In contrast, at −10 dB SIR, the NDL initially remains high but drops significantly as the penalty increases, reaching approximately −35 dB when the penalty ξ reaches approximately 1e5, suggesting enhanced nulling depth with increasing penalty values. For −20 dB SIR, the NDL follows a similar trend but starts at a lower level, quickly decreasing to around −45 dB when the penalty ξ reaches approximately 1e2, demonstrating that higher penalties significantly improve nulling performance under moderate interference. At the most challenging SIR level of −30 dB, the NDL begins near 0 dB and declines steeply as the penalty parameter rises, reaching about −50 dB when the penalty ξ reaches approximately 1e1. These results indicate that increasing the penalty parameter enhances the interference suppression capability of the system, especially under severe interference conditions, with progressively deeper nulls observed at lower SIR levels. Penalty plays a crucial role in optimizing the trade-off between exploration and exploitation, resulting in better interference suppression, particularly under high interference conditions.

Figure 4: The graph illustrates the relationship between the maximum SLL and the penalty ξ for different SIR levels (0 dB, −10 dB, −20 dB, −30 dB) using the BGWO algorithm. For SIR = 0 dB, SLL initially increases but stabilizes around −19 dB when the penalty ξ reaches approximately 1e4. For SIR = −10 dB and −20 dB, SLL also initially rises but stabilizes near −20 dB, respectively, when the penalty ξ is around 1e3. For SIR = −30 dB, SLL remains constant at −20 dB when the penalty ξ is around 1e1. These results show that the BGWO algorithm maintains stable control over the SLL as penalty ξ increases, but there is no significant improvement in reducing SLL as SIR decreases. SLL tends to decrease as the penalty ξ increases under high interference conditions, but it reaches a stable value after a certain threshold. This suggests that an optimal penalty ξ value is needed to balance between interference suppression and SLL control.

While $\xi$ has a noticeable positive effect on reducing NDL and improving interference suppression, its effect on SLL is more stable. The SLL only slightly changes with increasing $\xi$ and does not decrease much after reaching its threshold. Therefore, increasing $\xi$ tends to improve interference suppression without significantly impacting SLL. The results show a clear correlation between penalty values and interference suppression performance. Therefore, to achieve a balanced trade-off, a penalty value of $10$ will be selected for the remaining scenarios.

Figure 3: NDL at 25${}^{\circ }$ with different $\xi$ values.

Figure 4: Maximum SLL with different $\xi$ values.

B. Convergence characteristics

In this scenario, we evaluate the convergence speed of the BGWO algorithm and compare it with other binary metaheuristic algorithms by adaptively imposing nulls when an interference emerges at $\theta ={25}^{\circ }$. The three figures below provide insights into the performance of the algorithms in suppressing unknown interference.

Figure 5: The convergence characteristics of the fitness function over iterations for various population sizes using the BGWO algorithm reveal distinct patterns.With a population size of 10, the algorithm begins at a high initial fitness value, followed by a rapid decline and gradual convergence around iteration 20. Increasing the population size to 30 results in faster convergence, with stability achieved by iteration 10, while a population of 50 follows a similar pattern, stabilizing shortly after iteration 10. For a population size of 100, convergence occurs slightly more slowly, yet it reaches the same fitness level by iteration 5, as seen in larger populations.

The fitness value’s rapid decline with iteration count highlights the BGWO algorithm’s ability to converge swiftly towards an optimal scheme. Larger populations, specifically sizes $50$ and $100$, exhibit greater stability during convergence, with minimal oscillation after reaching a plateau. In contrast, smaller populations such as $10$ converge more gradually and show pronounced oscillation in the early iterations, although they ultimately attain comparable fitness levels to larger populations.

Beyond iteration 5, all population sizes converge to similar fitness values, indicating that increasing the population size beyond a certain threshold yields limited gains in solution quality. However, larger population sizes contribute to faster and more stable early convergence, underscoring their advantage in initial stabilization.

Figure 5: Convergence of fitness function with different population sizes using BGWO.

Figure 6: Based on the provided graph, we can observe a comparison between the BBA, BWOA, and BGWO algorithms in terms of the convergence of the fitness function over iterations.

• BBA: Starts with a higher fitness value compared to the other two algorithms but decreases quickly in the first few iterations and gradually converges to a stable state around iteration $10$.

• BWOA: Exhibits a faster initial decline than BBA and reaches a stable state around iteration $10$.

• BGWO: Similar to BWOA, BGWO drops rapidly and reaches stability fairly early, around iteration $5$. BGWO seems to converge faster and achieves the lowest fitness value among all the algorithms.

All algorithms show a sharp decline in fitness values during the initial iterations, but BGWO demonstrates the fastest and most stable convergence compared to BBA and BWOA. After around iteration $5-10$, all algorithms reach similar convergence levels; however, BGWO shows an advantage in terms of both speed and early stability. This suggests that BGWO could be the more efficient algorithm for optimizing the fitness function with fewer iterations compared to BBA and BWOA.

Figure 7: The graph compares the convergence of the fitness function between two versions of the BGWO algorithm: with and without optimal variable number minimization.

• BGWO with optimal variable number minimization: convergence speed is faster, with the fitness function rapidly decreasing from the first iterations and reaching a stable value after approximately $3$ iterations. The fitness function converges to a lower and more stable level compared to the method without variable number minimization.

• BGWO without optimal variable number minimization: convergence speed is slower, requiring more iterations to achieve a lower fitness value. It takes more than $10$ iterations to approach convergence, and the decrease in the fitness function is more gradual compared to the method with variable number minimization.

Figure 6: Comparison of SPM fitness functions based on three different optimization algorithms.

Figure 7: Comparison of fitness function of SPM based on BGWO with and without optimal variable number minimization.

The comparison shows that the BGWO method with optimal variable number minimization demonstrates better performance in terms of convergence speed and optimal scheme quality. This indicates that reducing the number of variables during optimization can enhance the algorithm’s performance, achieving better results in a shorter time.

C. Adaptive null-steering capability

This scenario illustrates the beamformer’s adaptive null-steering capability through the use of BBA, BWOA, and BGWO optimization algorithms based on the SPM. The population size is configured to $50$, and the maximum iteration limit is set to $3$. Figure 8 illustrates the optimized radiation patterns, assuming an interference emerges within a broad null from $\varphi ={30}^{\circ }\mathrm{to}{\mathrm{\hspace{0.17em}\hspace{0.17em}41}}^{\circ },$ while steering the main lobe towards $\theta ={90}^{\circ }$ and $\varphi =5^{\circ }$. The optimized patterns generated by BBA, BWOA, and BGWO based on SPM retain most of the characteristics of the Chebyshev pattern, including a half-power beamwidth of $6.4{}^{\circ }$ and SLL of $-25$ dB, except for the maximum SLL of $-19.3$ dB and a wide null range from $30{}^{\circ }$ to $41{}^{\circ }$. NDL reaches a minimum of approximately $-33$ dB for the BGWO, BBA, and BWOA patterns. The CDF (cumulative distribution function) plot of SLL between the Chebyshev $-25$ dB method and BGWO in Fig. 9 highlights these results, with the BGWO curve showing lower SLL values compared to the Chebyshev method for most SLL values. This indicates that BGWO reduces the SLL comparable to the Chebyshev $-25$ dB method in certain scenarios. From Fig. 9, it can be seen that the BGWO method not only optimizes the SLL but also reduces the interference levels to deeper negative values.

Figure 8: Adapted patterns with a broad null when steering the main lobe.

Figure 9: CDFs of SLLs for cases of BGWO with SPM and a broad null from $\varphi ={30}^{\circ }\mathrm{to}{\mathrm{\hspace{0.17em}\hspace{0.17em}41}}^{\circ }$.

To further evaluate the impact of SLL constraint on beamforming performance, we generated optimized radiation patterns with different SLL constraints, as shown in Fig. 10. The results demonstrate that when the SLL constraint is set to $-25$ dB, the proposed method effectively suppresses unknown interference while maintaining a narrow half-power beamwidth. When the constraint is further reduced to $-30$ dB, $-32$ dB, and $-34$ dB, suppression capability remains effective but the half-power beamwidth increases accordingly. This trade-off occurs because a deeper SLL constraint forces the optimization process to limit sidelobes more strictly, which in turn affects main lobe shaping. Additionally, due to the structural properties of Algorithm 1, the phase matrix in the optimization process exhibits odd symmetry, leading to a slight increase in SLL at the symmetric direction of the interference source. However, this effect is minimal and does not degrade the interference suppression capability of the proposed method.

Figure 10: Optimized radiation patterns with different SLL constraints $(-25$ dB, $-30$ dB, $-32$ dB, and $-34$ dB).

Table 1: Convergence time ($F(\mathbf{\text{w}},\xi )\le 220$) and maximum iterations for BGWO, BWOA, and BBA algorithms

Time (s) to Achieve	BGWO	BWOA	BBA
$F(\mathbf{\text{w}},\xi )\le 220$	0.0191	0.0127	0.0159
Maximum number of iterations is reached	0.0642	0.0570	0.0558

Table 1 presents convergence times for the three algorithms BGWO, BWOA, and BBA, measured over $1000$ Monte Carlo runs on MATLAB Online using an Intel(R) Xeon(R) CPU @ 2.20 GHz. Results show that BWOA has the fastest convergence time to the target value $F(\mathbf{\text{w}},\xi )\le 220,$ achieving it in $0.0127$ seconds, followed by BBA at $0.0159$ seconds, and BGWO at $0.0191$ seconds. Despite BGWO being slightly slower in this instance, its convergence to the scheme is still efficient. When the maximum number of iterations is reached, BGWO takes the longest at $0.0642$ seconds, while BWOA and BBA perform similarly, taking $0.0570$ seconds and $0.0558$ seconds, respectively. This highlights the trade-off between speed and performance consistency across different algorithms.

VI. CONCLUSION

This paper has proposed a scheme to address unknown interference suppression in AAS. First, we reduce the complexity by minimizing the number of optimization variables, significantly shortening computational time, which is an essential factor for efficient AAS deployment. Second, we integrate a CHT, specifically the SPM, with metaheuristic algorithms, including the BBA, BGWO, and BWOA, to improve interference suppression in URAs. The effectiveness of the scheme is validated through simulations, demonstrating strong convergence and the ability to suppress interference in sidelobe regions without prior knowledge of interference direction. These results underscore the scheme’s potential for unknown interference management in AAS. Future research should explore alternative CHT methods, additional array configurations, and control over multiple main lobes, as well as addressing interference in the main lobe region. Expanding this approach to broader challenges within AAS could further enhance its practical applications.

REFERENCES

[1] H. Asplund, J. Karlsson, F. Kronestedt, E. Larsson, D. Astely, P. von Butovitsch, T. Chapman, M. Frenne, F. Ghasemzadeh, M. Hagström, B. Hogan, and G. Jöngren, Advanced Antenna Systems for 5G Network Deployments: Bridging the Gap Between Theory and Practice. Cambridge, MA: Academic Press, 2020.

[2] N. Trabelsi, L. C. Fourati, and C. S. Chen, “Interference management in 5G and beyond networks: A comprehensive survey,” Computer Networks, vol. 239, 2024.

[3] C. A. Balanis, Antenna Theory: Analysis and Design, 4th ed. Oxford: John Wiley and Sons, 2016.

[4] L. T. Trang, N. V. Cuong, and T. V. Luyen, “Interference suppression approaches utilizing phase-only control and metaheuristic algorithms: A comparative study,”Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol. 558, 2024.

[5] T. V. Luyen and N. V. Cuong, “An adaptive beamformer utilizing Binary Bat Algorithm for antenna array pattern nulling,” Journal of Science and Technology, Hanoi University of Industry, vol. 57, pp. 52-57, 2021.

[6] J. Yin, Z. Liu, Y. Jin, D. Peng, and J. Kang, “Blind source separation and identification for speech signals,” in 2017 International Conference on Sensing, Diagnostics, Prognostics, and Control (SDPC), Shanghai, China, pp. 398-402, 2017.

[7] T. Oyedare, V. K. Shah, D. J. Jakubisin, and J. H. Reed, “Interference suppression using deep learning: Current approaches and open challenges,” IEEE Access, vol. 10, pp. 66238-66266, 2022.

[8] D. Chen, Y. Zhuang, J. Huai, X. Sun, X. Yang, and M. Awais Javed, “Coexistence and interference mitigation for WPANs and WLANs from traditional approaches to deep learning: A review,” IEEE Sensors J., vol. 21, no. 22, pp. 25561-25589, 2021.

[9] S. Wang, F. Han, Y. Yan, Y. Ding, P. Yang, and X.-Y. Li, “SlickScatter: Retrieve WiFi backscatter signal from unknown interference,” in 2024 IEEE/ACM 32nd International Symposium on Quality of Service (IWQoS), pp. 1-10, 2024.

[10] Q. Zhang, H. Ji, L. Li, and Z. Zhu, “Automatic modulation recognition of unknown interference signals based on graph model,” IEEE Wireless Communications Letters, vol. 13, no. 9, pp. 2317-2321, 2024.

[11] Z. Xu, T. Song, and X. Wei, “Unknown interference source positioning based on near-field scanning,” in 2021 IEEE MTT-S International Wireless Symposium (IWS), pp. 1-3, 2021.

[12] T.-Y. Huang, B. Lin, A. Ahmed, M.-Y. Huang, and H. Wang, “A 23-37-GHz autonomous 2-D MIMO receiver array with rapid full-FoV spatial filtering for unknown interference suppression,” IEEE Transactions on Microwave Theory and Techniques, vol. 71, no. 11, pp. 4841-4854, 2023.

[13] M. Zaher, E. Björnson, and M. Petrova, “A Bayesian approach to characterize unknown interference power in wireless networks,” IEEE Wireless Communications Letters, vol. 12, no. 8, pp. 1374-1378, 2023.

[14] I. P. Gravas, Z. D. Zaharis, T. V. Yioultsis, P. I. Lazaridis, and T. D. Xenos, “Adaptive beamforming with sidelobe suppression by placing extra radiation pattern nulls,” IEEE Transactions on Antennas and Propagation, vol. 67, no. 6, pp. 3853-3862, 2019.

[15] T. V. Luyen, H. M. Kha, N. V. Tuyen, and T. V. B. Giang, “An efficient ULA pattern nulling approach in the presence of unknown interference,” Journal of Electromagnetic Waves and Applications, vol. 35, no. 1, pp. 1-18, 2021.

[16] T. Luyen and C. Nguyen, “An effective beamformer for interference suppression without knowing the direction,” International Journal of Electrical and Computer Engineering (IJECE), vol. 13, no. 1, pp. 601-610, 2023.

[17] S. Mirjalili, S. M. Mirjalili, and X. S. Yang, “Binary Bat Algorithm,” Neural Computing and Applications, vol. 25, no. 3-4, pp. 663-681, 2014.

[18] A. G. Hussien, A. E. Hassanien, E. H. Houssein, S. Bhattacharyya, and M. Amin, “S-shaped binary Whale Optimization Algorithm for feature selection,” Advances in Intelligent Systems and Computing, vol. 727, 2019.

[19] E. Emary, H. M. Zawbaa, and A. E. Hassanien, “Binary Grey Wolf Optimization approaches for feature selection,” Neurocomputing, vol. 172, pp. 371-381, 2016.

[20] A. Sharma, “Antenna array pattern synthesis using metaheuristic algorithms: A review,” IETE Technical Review, pp. 1-26, Mar. 2022.

[21] N. D. Lagaros, M. Kournoutos, N. Ath. Kallioras, and A. N. Nordas, “Constraint handling techniques for metaheuristics: A state-of-the-art review and new variants,” Optim. Eng., vol. 24, pp. 2251-2298, 2023.

[22] X.-S. Yang, Nature Inspired Optimization Algorithms. Amsterdam: Elsevier, 2014.

[23] I. Rahimi, A. H. Gandomi, F. Chen, and E. Mezura-Montes, “A review on constraint handling techniques for population-based algorithms: From single-objective to multi-objective optimization,” Arch. Computat. Methods Eng., vol. 30, pp. 2181-2209, 2023.

[24] A. R. Jordehi, “A review on constraint handling strategies in particle swarm optimization,” Neural Comput. & Applic., vol. 26, pp. 1265-1275, 2015.

[25] Ö. Yeniay, “Penalty function methods for constrained optimization with genetic algorithms,” Math. Comput. Appl., vol. 10, no. 1, pp. 45-56, 2005.

[26] H. M. Kha, T. V. Luyen, and N. V. Cuong, “An efficient beamformer for interference suppression using rectangular antenna arrays,” J. Commun., vol. 18, no. 2, pp. 116-122, 2023.

[27] F. B. Gross, Smart Antennas with MATLAB, 2nd ed. New York, NY: McGraw-Hill Education Press, pp. 387-391, 2015.

BIOGRAPHIES

Le Thi Trang obtained both the B.E. degree and the M.S. degree from the Institute of Military Technology in 2007 and 2011, respectively. Her research interests primarily focus on beamformers, smart antennas, nature-inspired optimization algorithms, and constraint handling techniques.

Nguyen Van Cuong received the Engineer’s and master’s degrees from Hanoi University of Industry, in 2020 and 2022, respectively. His research interests include emerging wireless communication technologies, metaheuristics algorithms, deep learning, and convex optimization.

Hoang Thi Phuong Thao was born in Vietnam in 1981. She received the Diploma of Engineer (2004), Master of Science (2007), and Ph.D. degree (2019) in Electronics and Telecommunications from Hanoi University of Science and Technology, Vietnam. Her research interests are designing antenna, localization systems, and wireless communication technology.

Tong Van Luyen received the B.S. and M.S. degrees from the Hanoi University of Science and Technology, in 2002 and 2004, respectively, and Ph.D. degree from VNU University of Engineering and Technology in 2019. His research interests are in beamforming and beam steering for antenna arrays, smart antennas, optimum array processing, nature-inspired optimization algorithms, and artificial intelligence.

ACES JOURNAL, Vol. 40, No. 3, 192–202
doi: 10.13052/2025.ACES.J.400304
© 2025 River Publishers

ABSTRACT

I. INTRODUCTION

II. CONSTRAINED HANDLING TECHNIQUES AND CONSTRAINED OPTIMIZATION PROBLEMS

A. Constrained handling techniques

B. Constrained optimization problems

III. PROBLEM FORMULATION

IV. PROPOSED SCHEME

V. NUMERICAL RESULTS AND DISCUSSION

A. Effect of penalty parameter

B. Convergence characteristics

C. Adaptive null-steering capability

VI. CONCLUSION

REFERENCES

BIOGRAPHIES