You could go to court and fight for custody .. But at a disasvantage. It's very difficult to challenge a parent for custody. You would have to prove him unfit.
How can I as maternal grandmother gain custody of 3month grandchild despite my daughter willingly ceding Sole Custody&100% to the father b/c she prefers to be childfree&doesn't want illegitimate children?