Form-finding rules, when synthesized as a Structural Grammar, enable access to a vast and diverse Combinatorial Structural design space. Reinforcement Learning offers a unique opportunity to navigate this space and identify trajectories that yield novel and efficient solutions through self-play and reward maximization. Building on prior work integrating Combinatorial Equilibrium Modelling (CEM) with Deep Reinforcement Learning and Graph Neural Networks (GNNs), this study extends the grammar-based action space to three dimensions and develops a GNN-driven Deep Q-learning agent for sequential structural design under prescribed loads and support conditions. The central challenge is enabling the agent to generalize to diverse problem sets beyond its training distribution. We address this by focusing our study in two directions: Firstly, by leveraging an Equivariant Graph Neural Network architecture, the agent learns symmetry-aware node and edge embeddings that are invariant to geometric transformations such as translation, rotation, and reflection. This improves sample efficiency by avoiding redundant representations and reducing the action space. Secondly, to promote generalisation to unseen and more complex design tasks, we employ a curriculum learning strategy that gradually increases task complexity, thereby enabling the agent to acquire transferable design policies. Through design case studies and ablation studies, we evaluate the agent’s learned policies and analyse the impact of individual architectural and algorithmic choices. We envision this framework as a powerful tool for early-stage structural design, in which an intelligent agent capable of form-finding through a grammar-based ruleset can meaningfully support and collaborate with architects and engineers.