Distributed online convex optimization (D-OCO), where multiple agents within a network collaboratively learn optimal decisions in real-time, arises naturally in applications such as federated learning, sensor networks, and multi-agent control. In this paper, we study D-OCO under unknown, time- and agent-varying feedback delays. While recent work has addressed delayed feedback (Nguyen, Kim Thang, and Trystram 2024), existing algorithms assume prior knowledge of the cumulative delay over agents and still suffer from suboptimal dependence on both the delay and network parameters. To overcome these limitations, we propose a novel algorithm that achieves an improved regret bound of $\widetilde{\mathcal{O}}\Big(\sqrt{N^3d{\text{tot},T}} + \sqrt{\frac{N^{5/2}T}{\sqrt{1-\sigma_2}}}\Big)$, where $d{\text{tot}}$ denotes the average total delay across agents, $N$ is the number of agents, and $1 - \sigma2$ is the spectral gap of the network. We also show that the dependence on $T$, $d{\text{tot}}$ and $\sigma_2$ is tight by providing a matching lower bound. Our approach builds upon recent advances in D-OCO (Wan et al. 2024), but crucially incorporates an adaptive learning rate mechanism via a decentralized communication protocol. This enables each agent to estimate delays locally using a gossip-based strategy without the prior knowledge of the total delay. We further extend our framework to the strongly convex setting and derive sharper regret bounds. Experimental results validate the effectiveness of our approach, showing improvements over existing benchmark algorithms.